U.S. DEPARTMENT OF COMMERCE • National Telecommunications and Information Administration

report series

NTIA Report 13-495

Intelligibility of Analog FM and Updated 

P25 Radio Systems in the Presence of 

Fireground Noise: Test Plan and Results

David J. Atkinson

Andrew A. Catellier

U.S. DEPARTMENT OF COMMERCE

May 2013

NTIA Report 13-495

Intelligibility of Analog FM and Updated 

P25 Radio Systems in the Presence of 

Fireground Noise: Test Plan and Results

David J. Atkinson

Andrew A. Catellier

iii

DISCLAIMER

Certain commercial equipment and materials are identified in this report to specify adequately 

the technical aspects of the reported tests and results. In no case does such identification imply 

recommendation or endorsement by the National Telecommunications and Information 

Administration, nor does it imply that the material or equipment identified is the best available 

for this purpose.
iv
v

CONTENTS

Figures.......................................................................................................................................... viii

Tables...............................................................................................................................................x

Abbreviations/Acronyms............................................................................................................. xiii

Executive Summary.................................................................................................................... xvii

1 Introduction...................................................................................................................................1

1.1 Project 25 Standardization .....................................................................................................2

1.2 Scope of this Report...............................................................................................................3

1.3 Overview................................................................................................................................3

2 Test Elements................................................................................................................................7

2.1 Input and Playback Systems..................................................................................................8

2.2 Reference Systems.................................................................................................................9

2.2.1 Project 25 Reference Systems...................................................................................10

2.2.2 25 kHz AFM Reference System ...............................................................................11

2.2.3 12.5 kHz AFM Reference System ............................................................................13

2.3 Acoustic Environments........................................................................................................15

2.4 Channel Impairments...........................................................................................................20

2.5 Test Conditions....................................................................................................................21

3 Speech Data Bases......................................................................................................................22

3.1 Speech Data Base Properties ...............................................................................................22

3.2 Speech Data Base Levels.....................................................................................................23

3.3 Signal and Acoustic Noise Levels .......................................................................................24

4 Production of Recorded Files......................................................................................................25

4.1 Test Elements.......................................................................................................................25

4.2 Test Signal Preparation........................................................................................................25

5 Subjective Evaluation of Intelligibility.......................................................................................28

5.1 Stimulus File Preparation.....................................................................................................28

5.2 MRT Evaluation Laboratory................................................................................................28

5.3 Intelligibility Testing ...........................................................................................................29

5.4 Analysis and Reporting of Results.......................................................................................30

6 Test Results.................................................................................................................................32

6.1 Experiment Results by Scenario ..........................................................................................32

6.1.1 Results for Scenario 1: “No Noise, No Mask, Clean RF Channel”..........................32

6.1.2 Results for Scenario 2: “No Noise, Mask (Vox Port), Clean RF Channel”..............35

6.1.3 Results for Scenario 3: “No Noise, Mask (Int Mic), Clean RF Channel” ................37

6.1.4 Results for Scenario 4: “PASS 1, Mask (Vox Port), Clean RF Channel” ................39

6.1.5 Results for Scenario 5: “PASS 2, Mask (Vox Port), Clean RF Channel” ................41

6.1.6 Results for Scenario 6: “PASS 1, Mask (Int Mic), Clean RF Channel”...................43
vi

6.1.7 Results for Scenario 7: “PASS 2, Mask (Int Mic), Clean RF Channel”...................45

6.1.8 Results for Scenario 8: “Night Club Noise, No Mask, Clean RF Channel”.............47

6.1.9 Results for Scenario 9: “No Noise, No Mask, Degraded RF Channel”....................49

6.1.10 Results for Scenario 10: “No Noise, Mask (Vox Port), Degraded RF 

Channel”..................................................................................................................51

6.1.11 Results for Scenario 11: “No Noise, Mask (Int Mic), Degraded RF 

Channel”..................................................................................................................53

6.1.12 Results for Scenario 12: “PASS 1, Mask (Vox Port), Degraded RF 

Channel”..................................................................................................................55

6.1.13 Results for Scenario 13: “PASS 1, Mask (Int Mic), Degraded RF Channel”.........57

6.1.14 Results for Scenario 14: “Night Club, No Mask, Degraded RF Channel” .............58

6.2 Additional Comparisons of Results.....................................................................................59

6.2.1 Comparison of Two PASS Alarms...........................................................................59

6.2.2 Comparison of SCBA Masks....................................................................................61

6.2.3 Differences Between Previous and Current Test......................................................65

6.3 Summary of Observations ...................................................................................................68

7 Conclusion ..................................................................................................................................69

8 Acknowledgements.....................................................................................................................70

9 References...................................................................................................................................71

Appendix A : Condition Labels and Minitab Results....................................................................73

A.1 Minitab Report for Scenario 1 ............................................................................................75

A.2 Minitab Report for Scenario 2 ............................................................................................76

A.3 Minitab Report for Scenario 3 ............................................................................................77

A.4 Minitab Report for Scenario 4 ............................................................................................78

A.5 Minitab Report for Scenario 5 ............................................................................................79

A.6 Minitab Report for Scenario 6 ............................................................................................80

A.7 Minitab Report for Scenario 7 ............................................................................................81

A.8 Minitab Report for Scenario 8 ............................................................................................82

A.9 Minitab Report for Scenario 9 ............................................................................................83

A.10 Minitab Report for Scenario 10 ........................................................................................84

A.11 Minitab Report for Scenario 11 ........................................................................................85

A.12 Minitab Report for Scenario 12 ........................................................................................86

A.13 Minitab Report for Scenario 14 ........................................................................................87

A.14 Minitab Report for PASS Alarm Comparison Using Vox Port........................................88

A.15 Minitab Report for PASS Alarm Comparison Using Internal Microphone .....................91

A.16 Minitab Report for SCBA Comparison with no Background Noise ................................94

A.17 Minitab Report for SCBA Comparison with PASS 1 Background Noise........................97

A.18 Minitab Report for SCBA Comparison with PASS 2 Background Noise......................100

A.19 Minitab Report for SCBA Comparison with Degraded Channel and 

No Background Noise ....................................................................................................103

Appendix B : Listener Scores......................................................................................................106
vii

Appendix C : Source Speech Recording Information..................................................................118

Appendix D : Listening Laboratory Configuration .....................................................................119

Appendix E : Comparison of Talker Versus a Head and Torso Simulator for Intelligibility 

Testing..........................................................................................................................................122
viii

FIGURES

Figure 1. Test plan overview. ..........................................................................................................6

Figure 2. Schematic overview of input, reference, and playback systems. .....................................7

Figure 3. Input system for test. Solid lines represent electrical paths, dotted lines 

represent acoustic paths...................................................................................................8

Figure 4. Output system for test.......................................................................................................9

Figure 5. P25 reference systems configuration..............................................................................11

Figure 6. 25 kHz AFM reference system configuration. ...............................................................12

Figure 7. 12.5 kHz AFM reference system configuration. ............................................................14

Figure 8. PSD plot and spectrogram of two PASS alarms from manufacturer 1 sounding. 

Spectrogram has x-axis in seconds and y-axis in Hz. ...................................................17

Figure 9. PSD plot and spectrogram of two PASS alarms from manufacturer 2 sounding. 

Spectrogram has x-axis in seconds and y-axis in Hz. ...................................................18

Figure 10. PSD plot and spectrogram of night club noise. Spectrogram has x-axis in 

seconds and y-axis in Hz...............................................................................................19

Figure 11. Acoustic path loss through SCBA mask. .....................................................................20

Figure 12. Physical configuration showing acoustic anechoic chamber, speakers, 

microphone, and HATS for non-mask conditions.........................................................26

Figure 13. Physical configuration showing acoustic anechoic chamber, speakers, SCBA 

mask, microphone, and HATS for conditions with SCBA mask and external 

microphone....................................................................................................................27

Figure 14. Physical configuration showing acoustic anechoic chamber, speakers, SCBA 

mask with internal microphone, and HATS for conditions with SCBA mask 

and internal microphone................................................................................................27

Figure 15. Intelligibility scores (RA) for Scenario 1. .....................................................................34

Figure 16. Intelligibility scores (RA) for Scenario 2. .....................................................................36

Figure 17. Intelligibility scores (RA) for Scenario 3. .....................................................................38

Figure 18. Intelligibility scores (RA) for Scenario 4. .....................................................................40

Figure 19. Intelligibility scores (RA) for Scenario 5. .....................................................................42
ix

Figure 20. Intelligibility scores (RA) for Scenario 6. .....................................................................44

Figure 21. Intelligibility scores (RA) for Scenario 7. .....................................................................46

Figure 22. Intelligibility scores (RA) for Scenario 8. .....................................................................48

Figure 23. Intelligibility scores (RA) for Scenario 9. .....................................................................50

Figure 24. Intelligibility scores (RA) for Scenario 10. ...................................................................52

Figure 25. Intelligibility scores (RA) for Scenario 11. ...................................................................54

Figure 26. Intelligibility scores (RA) for Scenario 12. ...................................................................56

Figure 27. Intelligibility scores (RA) for Scenario 13. ...................................................................57

Figure 28. Intelligibility scores (RA) for Scenario 14. ...................................................................59

Figure 29. Intelligibility scores for PASS 1 and PASS 2 for a mask with a vox port. ..................60

Figure 30. Intelligibility scores for PASS 1 and PASS 2 for a mask with internal mic. ...............61

Figure 31. Intelligibility scores for SCBA mask with a vox port or internal microphone. ...........62

Figure 32. SCBA mask comparison with PASS 1 background noise............................................63

Figure 33. SCBA mask comparison with PASS 2 background noise............................................63

Figure 34. SCBA mask comparison with degraded channel. ........................................................64

Figure 35. SCBA mask comparison with degraded channel and PASS 1 background 

noise...............................................................................................................................64

Figure 36. Graphical representation of intelligibility scores achieved in three different 

scenarios by AFM systems in both the previous and current test. ................................66

Figure 37. Graphical representation of intelligibility scores achieved in three different 

scenarios by P25 systems in both the previous and current test....................................67

Figure D-1. Diagram showing physical layout of test chambers.................................................120

Figure D-2. Real time spectrum analysis of pink noise in Chamber 1. .......................................121

Figure D-3. Real time spectrum analysis of pink noise in Chamber 2. .......................................121

Figure E-1. Bar chart of talker vs. HATS intelligibility scores. ..................................................123
x

TABLES

Table 1. Simulated degraded static RF channel conditions. ............................................................5

Table 2. Degraded static RF channel conditions. ..........................................................................20

Table 3. Combinations of scenarios and communication systems. ...............................................21

Table 4. Source audio arrangement of sentences, test conditions, and talkers. .............................23

Table 5. Intelligibility scores, ANOVA results, and Tukey results for Scenario 1. ......................33

Table 6. Intelligibility scores, ANOVA results, and Tukey results for Scenario 2. ......................35

Table 7. Intelligibility scores, ANOVA results, and Tukey results for Scenario 3. ......................37

Table 8. Intelligibility scores, ANOVA results, and Tukey results for Scenario 4. ......................39

Table 9. Intelligibility scores and ANOVA results for Scenario 5. ...............................................41

Table 10. Intelligibility scores, ANOVA results, and Tukey results for Scenario 6. ....................43

Table 11. Intelligibility scores, ANOVA results, and Tukey results for Scenario 7. ....................45

Table 12. Intelligibility scores, ANOVA results, and Tukey results for Scenario 8. ....................47

Table 13. Intelligibility scores, ANOVA results, and Tukey results for Scenario 9. ....................49

Table 14. Intelligibility scores, ANOVA results, and Tukey results for Scenario 10. ..................51

Table 15. Intelligibility scores, ANOVA results, and Tukey results for Scenario 11. ..................53

Table 16. Intelligibility scores, ANOVA results, and Tukey results for Scenario 12. ..................55

Table 17. Intelligibility scores for Scenario 13..............................................................................57

Table 18. Intelligibility scores and ANOVA results results for Scenario 14.................................58

Table 19. Intelligibility scores for PASS alarm comparison .........................................................60

Table 20. Intelligibility scores for comparing vox port to internal microphone............................62

Table 21. Experiment configuration details for the previous and current MRT............................65

Table 22. Intelligibility scores achieved in three different scenarios by AFM radios in 

both the previous and current test..................................................................................66
xi

Table 23. Intelligibility scores achieved in three different scenarios by P25 systems in 

both the previous and current test..................................................................................66

Table A-1 List of condition labels used for this experiment. ........................................................73

Table B-1. Listener scores by talker, condition, and listener.......................................................106

Table E-1. Intelligibility scores for talker versus HATS .............................................................122
xii
xiii

ABBREVIATIONS/ACRONYMS

A-to-D Analog to digital conversion

AFM Analog frequency modulation

ANOVA Analysis of variance

APCO Association of Public Safety Communications Officials.

APWG Audio Performance Working Group

BER Bit error ratio—the ratio of errored bits to total bits in a sample

Codec Combination of an encoder and decoder in series (encoder/decoder).

Coder Same as encoder

C4FM Compatible 4 level frequency modulation

CNR Carrier to noise ratio

CRC Cyclic redundancy check

CTCSS Continuous tone-coded squelch system—a mechanism that allows multiple 

groups to use a common frequency without having to hear the other groups’ 

messages

D-to-A Digital to analog conversion

dB Decibel

dBA Sound pressure level in dB measured using the A-weighted scale.

dBC Sound pressure level in dB measured using the C-weighted scale.

dBov Decibels relative to overload point

Decoder Device for translation of a signal from a digital representation into an analog 

format (for the purposes of this document, this is a device compatible with [1]

DF ANOVA variable: degrees of freedom (also used for Tukey Multiple Comparison 

Test calculations)

DHS U.S. Department of Homeland Security

DSP Digital signal processor

DVSI Digital Voice Systems, Inc.

ENBW Effective noise bandwidth

Enhanced An updated version of the Project 25 vocoder defined in [1] with several 

enhancements, including soft-decision decoding, noise suppression, updated 

spectral masks, and echo cancellation

Encoder Device for converting an analog signal into a digital representation. For the 

purposes of this document, this is a device compatible with [1]

F ANOVA variable: f-statistic

FIR Finite impulse response filter

FM Frequency modulation

FPIC Federal Partnership for Interoperable Communications

H A hypothesis under evaluation
xiv

HATS Head and torso simulator

HCPM Harmonized continuous phase modulation

HDQPSK Harmonized differential quadrature phase shift keying modulation

HPF High pass filter

IAFC International Association of Fire Chiefs

IF Intermediate frequency

ITU-T International Telecommunication Union – Telecommunication Standardization 

Sector

LLR Log likelihood ratio

LPF Low pass filter

LRP Lip reference point of the HATS

MATLABTM Mathematical analysis software by Mathworks, Inc.

MRT Modified Rhyme Test as defined in [9]

MS ANOVA variable: mean squares

NC noise criteria

NFPA National Fire Protection Association.

n A value used as an index.

OBSA Octave Band Set A

OBSB Octave Band Set B

P ANOVA variable: value

P25 FR Project 25 full-rate vocoder

P25 HR Project 25 half-rate vocoder

PASS Personal alert safety system, which emits a signal in the event that the user 

becomes incapacitated or needs assistance

PC Personal computer

PCM Pulse code modulation, a logarithmically companded and 64 kb/s encoded 

representation of speech

PNG Pink noise generator

PSD Power spectral density

PTT Push to talk

RA Adjusted average intelligibility score

RF Radio frequency

RMS Root mean squared, the square root of the some of the mean values squared

SCBA Self-contained breathing apparatus

SINAD The ratio of signal to the summation of noise and distortion

SNR Signal-to-noise ratio

SS ANOVA variable: sum of squares

TIA Telecommunications Industry Association

vocoder Voice encoder/decoder
xv

vox Voice operated transmission

WAV Waveform Audio File Format, a commonly used high-quality digital audio file 

format for storing high quality audio bitstreams on PCs
xvi
xvii

EXECUTIVE SUMMARY

In 2008, the Institute for Telecommunication Sciences (ITS) developed and conducted a test 

designed to measure the intelligibility of land mobile radio (LMR) systems in the presence of 

fireground noise. The test showed that fireground noise environments were especially 

challenging for the digital speech coder component of the Project 25 (P25) LMR system. ITS 

published a report describing the test [6]. This report was distributed widely among the public 

safety community and resulted in the establishment of a set of new best practices that were 

intended to increase intelligibility performance in extreme conditions [7] [8]. Additionally, the 

digital speech coder used in P25 LMR systems was enhanced in order to address operation in 

typical fireground noise environments.

In order to measure the effect of the combination of these new best practices and updated voice 

coder, ITS conducted a new test. The goal of this test was to compare the performance of three 

LMR systems in both noisy and quiet environments with and without radio channel impairments. 

The test compared four systems:

 Project 25 reference system with updated digital speech coder (full rate)

 Project 25 reference system with updated digital speech coder (half rate)

 25 kHz Analog FM reference system

 12.5 kHz Analog FM reference system

The test simulated the use of those three systems in six different noise environments:

 No noise, no mask

 No noise, mask vox port

 No noise, mask internal microphone

 Personal alert safety systems 1 and 2, mask vox port

 Personal alert safety systems 1 and 2, mask internal microphone

 Night club noise, no mask

The design of the test was informed by the design used in the 2008 test, and used a Modified 

Rhyme Test (MRT) to measure each reference system’s intelligibility performance. An MRT is a 

test where a subject is asked to identify the word spoken at the end of a carrier sentence. In this 

case, a subject would hear the carrier sentence “Please select the word” followed by one of six 

rhyming words, for example: “bed,” “led,” “fed,” “red,” “wed,” “shed.” After hearing the 

sentence, the subject was then asked select the word that was spoken from a list presented 

through a graphical user interface (GUI). 

To test the intelligibility of each reference system, recorded MRT sentences were played back

through the mouth of a head and torso simulator (HATS), either directly into a microphone, 

through the vox port on a self-contained breathing apparatus (SCBA) and then into a 

microphone, or through the microphone included on an SCBA; acoustically mixed with different 

noise environments; and stored in high-fidelity audio files. These audio files were then processed 

by each reference system with and without simulated radio channel impairments. This resulted in

a pool of 67,200 processed speech files that were used as stimuli during the MRT.
xviii

Fifty-two subjects participated in the test, each listening to a specific subset of 2,400 of all the 

processed speech files. Analysis of the test results showed that the combination of new best 

practices and the updated digital voice coder result in improved speech intelligibility 

performance when compared to the 2008 test. When tested with clean radio channels, analog

systems generally performed marginally better but sometimes statistically significantly better. 

When each system was tested with a noisy radio channel, P25 systems performed better than 

analog systems in nearly every scenario. Both versions of the reference P25 system were found 

to be statistically similar in each scenario.
INTELLIGIBILITY OF ANALOG FM AND UPDATED P25 RADIO SYSTEMS IN THE 

PRESENCE OF FIREGROUND NOISE: TEST PLAN AND RESULTS

David J. Atkinson, Andrew A. Catellier1

This report describes a modified rhyme test (MRT) conducted to characterize the 

behavior of digital and analog communication in the presence of background 

noise and moderate RF channel degradation. This is done through the use of 

reference systems to provide a manufacturer-independent perspective on this 

issue.

Key words: intelligibility; Project 25; vocoder; modified rhyme test; noise; analog FM; land 

mobile radio; LMR; public safety; fire service

1 INTRODUCTION

This report describes the evaluation procedure used to characterize and compare the response of 

digital and analog voice communication technology to public safety noise environments. 

Understanding the characteristics of a digital voice coder is essential to enabling effective 

communication in the environment in which public safety personnel must operate. 

The evaluation procedure was designed to characterize and compare digital and analog 

communication technology in laboratory representations of public safety communications 

environments. Speech and noise from these environments were tested in a manufacturerindependent manner using reference communication systems. Reference communication systems

also enabled the testing of communications technology. This is accomplished through the 

examination of the system performance in a subjective listening test in which the relative 

performance among the systems is measured in a quantitative and repeatable way. 

The purpose of the subjective listening test was to evaluate reference systems under a variety of 

operating conditions. The operating conditions were chosen to be representative of those 

expected to be experienced in a public safety environment. Only a limited number of operating 

conditions were tested. To test all possible operating conditions would lead to a test that would 

be too unwieldy to conduct.

The test was designed to measure intelligibility through communication technologies being used 

in background noise situations. The test plan was developed in conjunction with the Audio 

Performance Working Group (APWG) and tests were conducted at the Public Safety 

 

1 The authors are with the Institute for Telecommunication Sciences, National Telecommunications and Information 

Administration, U.S. Department of Commerce, Boulder, CO 80305.
2

Communications Research (PSCR)2Laboratories in Boulder, Colorado, by researchers from the 

Institute for Telecommunication Sciences (ITS), National Telecommunications and Information 

Administration (NTIA). 

1.1 Project 25 Standardization

Project 25 is a public-private partnership established in 1989 by government entities and the 

Association of Public-Safety Communications Officials – International (APCO) for the primary 

purpose of realizing the benefits of digital narrowband land mobile radio (LMR) technologies for 

public safety practitioners and other users. Public safety, government, and manufacturer 

representatives participate in the P25 process to develop voluntary consensus standards with the 

support of the American National Standards Institute (ANSI)-accredited Telecommunications 

Industry Association (TIA). The goal of P25 is to specify formal standards for interfaces between 

the various components of an LMR system, commonly used by emergency responders, to enable 

easy interoperability of radios and other components, regardless of manufacturer.

The Project 25 (P25) vocoder standard was selected in 1992. At that time, several tests were 

conducted by P25 committees to ensure that the best available vocoder was selected. The 

selected vocoder, known as the baseline full-rate vocoder, was standardized by TIA as TIA102.BABA [1]. Further work continued on the development of the vocoder, and a dual-rate 

version of the vocoder was adopted in 2009 (TIA-102.BABA-1) [2] supported by additional tests 

(TSB-102.BABE [3], TSB-102.BABF [4]). Additionally, TIA adopted standard performance 

tests (TIA-102.BABG [5]) to ensure implemented vocoder-based systems have certain 

performance characteristics revealed by those tests. 

As P25 networks were deployed, there was a growing recognition that there were certain noisy 

environments that were problematic for the digital technologies. Of course, public safety

practitioners can’t simply choose not to operate in these noisy environments. This has appeared 

most consistently in tactical fireground communications, and has been raised to the national level 

by agencies such as Boise [Idaho] Fire, Fairfax [Virginia] Fire, Littleton [Colorado] Fire, and 

Phoenix [Arizona] Fire. The International Association of Fire Chiefs (IAFC) created a Digital 

Project Working Group (DPWG) to perform initial investigations into this issue. 

The work of the DPWG resulted in publication of the results of an intelligibility test as an NTIA 

technical report [6], the release of an IAFC Interim Report and Recommendations [7], and 

publication of Portable Radio Best Practices [8] by the IAFC. In response to this issue, the 

APCO Project 25 Interface Committee (APIC) created the Audio Performance Working Group 

(APWG) to perform testing to specifically quantify the problem and identify potential solutions,

and to develop testing methods that could be used to establish performance of communication 

technologies. These recommendations and potential solutions would be provided as 

recommendations to APIC and APIC Task Groups.

 

2 The PSCR program is a joint effort between the National Institute of Standards and Technology/ Law Enforcement 

Standards Office (NIST/OLES) and NTIA/ITS sponsored by the Department of Homeland Security, Office for 

Interoperability and Compatibility and Office of Emergency Communications, to advance public safety 

communications interoperability.
3

1.2 Scope of this Report

This report describes the procedures used to characterize and compare the intelligibility behavior 

of technologies using both analog communications and the TIA 102.BABA [1] compatible 

speech codec in environmental noise, moderately degraded channel conditions, and using two 

different self-contained breathing apparatus (SCBA) masks. The original baseline speech codec 

from 1992 is the baseline full rate system described in [1]. The speech codec is used to digitally 

encode the speech signal and provide forward error control for transmission at a data rate of 

7200 b/s. 

This vocoder was updated in 2003 to what became known as the enhanced dual rate vocoder, 

featuring bit rates of 7200 b/s and 3600 b/s. This version of the vocoder was subject to testing 

from 2003 to 2007 and performance measurement methods for the enhanced version of the 

vocoder are included in [5]. A further updated version of the codec was subsequently released in 

September 2009, referred to as version 1.60,3and this is the codec for the Project 25 Reference 

System described in Section 2. This version of the vocoder was chosen because there are a 

limited number of conditions that can be practically included in the test and because it represents 

the state-of-the-art of P25 vocoders current at the time of the tests. As such, it was expected to 

provide the best performance of the P25 vocoder family. 

The intelligibility evaluation of the digital and analog technologies under noise and degraded 

channel conditions is based on ANSI S3.2 and NFPA 1981 methods for speech intelligibility 

measurement [9] [10]. The speech preparation, processing, and presentation methods used in 

prior TIA subjective listening tests are appropriate for this intelligibility test and are listed in [6], 

[10] and [11].

This report takes the approach of comparing a reference system incorporating the speech codec 

with reference systems using analog frequency modulation (AFM). The experiment compared

the coding mechanisms with various background noise and channel conditions as might occur on 

a land mobile radio channel.

1.3 Overview

During this test, the performance of a reference implementation of a P25 radio system was 

compared to reference implementations of AFM systems. Specifically, the test evaluated the 

intelligibility of these reference systems using selected background noise and channel conditions. 

The test results were then compared among the reference systems. There were four reference 

systems in this test. These reference systems are listed below and more fully described in 

Section 2.

1. Project 25 Full-Rate (P25 FR) Reference System with vocoder DSP version 1.60 (software 

version 1.40e)

 

3 Version 1.60 refers to the Digital Signal Processor (DSP) code version of the vocoder. According to Digital Voice 

Systems, Inc., this is equivalent to PC executable floating point version 1.40e, which is the software used to conduct 

this test.
4

2. Project 25 Half-Rate (P25 HR) Reference System with vocoder DSP version 1.60 (software 

version 1.40e)

3. 25 kHz AFM Reference System

4. 12.5 kHz AFM Reference System

The intelligibility of each system was subjectively rated in each environmental condition 

specified. 

The intelligibility of a communication technology can be difficult to quantify since it is a 

subjective issue, relying on humans to be able to discern words. This performance evaluation 

relied on subjective testing using a panel of listeners who listened to speech passing through a 

system and attempted to understand what was said. Since discernment of listeners can vary, 

results were obtained from a number of listeners and averaged to obtain an overall score. 

To evaluate the intelligibility it was necessary to conduct an experiment in a controlled manner 

so that unintentional variation in the scoring could be avoided. The purpose of the testing was to 

characterize and compare the behavior of analog and digital voice communication technologies 

in public safety noise environments. The confidence we have that any apparent differences in 

performance were due to communication system effects and not random statistical variation 

depends upon how well we prevented differences from occurring in the testing. 

The statistical controls for the experiment and the analysis are given in Section 5. The listening 

test evaluated the reference systems under operating conditions, particularly different acoustic 

background noise conditions and different radio channel conditions. The acoustic background 

noise conditions were created from high-quality recordings of selected noisy acoustic 

environments encountered by LMR users in the fireground, reproduced in a laboratory 

environment. The radio channel conditions included an ideal channel and a moderately degraded 

channel and were simulated via software. This experiment used the following environmental 

noise conditions. Section 2.3 provides additional information about the background noises.

1. No background noise, no mask (referred to as the “clean” condition)

2. Mask voice port with no background noise

3. Mask with internal microphone with no background noise

4. Two manufacturers’ personal alert safety system (PASS) alarms, with mask using voice port

5. Two manufacturers’ PASS alarms, with mask using internal microphone

6. Night club, without mask

The experiment also used simulations of ideal and degraded channel conditions. The simulated 

degraded channel conditions are listed in Table 1. The noise conditions shown are static, not 

faded, and both ideal and degraded channels were applied to the noise conditions listed above.
5

Table 1. Simulated degraded static RF channel conditions.

System RF Level IF Noise Bandwidth CNR

25 kHz Analog -117 dBm 13.0 kHz 6.9 dB

12.5 kHz Analog -117 dBm 8.0 kHz 9.0 dB

P25 FR -117 dBm 5.8 kHz 10.4 dB

P25 HR -117 dBm 6.0 kHz 10.2 dB

In Section 2, Figure 2 shows a schematic of the three major stages for generating the processed 

files that were presented to the listener: generating high-quality recordings of speech mixed with 

noise, processing the recordings with software-based reference systems, and finally generating 

high-quality recordings of the result. Note that the P25 FR and P25 HR used the same executable 

code, just with different flags to invoke the proper rate. The reference system output files were 

used for playback to the listener, who then attempted to understand the spoken words. The input, 

playback and reference systems are described in Section 2

The overall plan of the test is outlined in Figure 1. The test began with the source audio material. 

There were several conditions placed on the source material, and these are covered in Section 3. 

The source audio material speech files were then acoustically mixed with the environmental 

noise conditions in a sound isolation chamber and recorded to files on a computer. Those files 

were then processed by the four reference systems, with different simulated channel conditions, 

which produced numerous output audio files. This procedure is given in Section 4. The output 

audio files were then organized into randomized blocks in order to provide samples suitable for a 

listening test. The randomization step, the listening test, and the analysis are described in Section 

5. The result of the test is then presented in Section 6, which also describes a software tool used 

for this analysis.
6

Get source audio material (Section 3)



Process the source material with the different noise 

conditions, channel impairments and reference systems 

(Section 4)



Randomize the order of the output audio samples 

(Section 5)



Conduct listening test experiments on the audio samples 

to get intelligibility scores. (Section 5)



Statistically analyze the intelligibility scores to detect 

any significant variations. (Section 5)



Present the results in a spreadsheet. (Section 6)

Figure 1. Test plan overview.
7

2 TEST ELEMENTS

Figure 2 shows a high-level schematic of the three major stages for generating the processed files 

that were presented to the listener. The first stage, or input system, was used to generate 

recordings (WAV files) of mixed speech and noise. The mixed speech and noise was then 

processed through the software-based reference systems (with or without channel impairments) 

and again recorded to WAV files. Note that the P25 FR and P25 HR used the same executable 

code with different flags to invoke the full-rate and half-rate options. The reference system 

output files were used for playback to the listener, who then attempted to understand the spoken 

words. Each of these components is described in more detail in the sections below.

Acoustically mix speech and background noise Speech Background Noise Input System

Record to file

P25 FR Reference System

25 kHz Analog Ref System 12.5 kHz Analog Ref System

Mixed Speech and Noise

Mixed Speech and Noise

Channel Impairment

Channel Impairment Channel Impairment Speech and Noise Processed Through Reference Systems

Speech and Noise Processed Through Reference Systems

Reference Systems

Acoustically mix processed material and pink noise Pink Noise Playback System

Play to Listener

P25 HR Reference System Channel Impairment

Figure 2. Schematic overview of input, reference, and playback systems.
8

2.1 Input and Playback Systems

The input and playback systems were used to record and play back, respectively, the audio 

samples for this test. A schematic diagram of the input system is shown in Figure 3; the system 

produced audio files that were sampled at 16 bits per sample and 48000 samples per second. The 

audio recorded consisted of source speech samples played through a head and torso simulator 

(HATS) into the acoustic environments listed in Section 1.3. This procedure is detailed in 

Section 4.2. The files were then processed through the reference systems and played out through 

the output system shown below.

HATS Speaker, No Mask External Mic HATS Speaker, Mask External Mic via Voice Port HATS Speaker, Mask Internal Mask MicRecorded Source Speech Acoustic Environmental Noise

PC Audio Recorder Input Gain A/D

Input Gain A/D External Mic

Figure 3. Input system for test. Solid lines represent electrical paths, dotted lines represent 

acoustic paths.

Characteristics associated with the input system are listed below:

1. A-to-D is ≥ 16 bits, ≥ 85 dB SNR

2. Input gain was set to provide active speech record level of approximately -22 dBm0

3. PC was used to record, store and play audio files

4. Ensure hum and noise were down by at least 60 dB4

5. External microphone was a Beyerdynamics MCD-100 cardioid electret microphone 

representing a high-quality version of a lapel microphone

6. Internal microphone was provided and integrated by the mask manufacturer

 

4 This specification was proposed by an APWG member and agreed to by the APWG as part of the discussion 

during experiment planning.
9

The playback system is shown in Figure 4, and was used to present the audio test samples (i.e., 

the output of the reference systems) to the listener. Note that the listening environment 

(described in Section 5.3 and Appendix D) also included a pink noise generation system.

PC Audio Recorder

Output

Gain

D-to-A Speaker

Figure 4. Output system for test.

Characteristics associated with the output system are listed below:

1. D-to-A is ≥ 16 bits, ≥ 85 dB SNR

2. Output gain set to 84 dBA5at Listener Reference Point.

3. Ensure hum and noise were down by at least 60 dB and preferably by at least 80 dB

4. Speaker/amplifier equalized to flat +- 3dB from 50 Hz to 10 kHz

2.2 Reference Systems

There are four reference systems defined for this test.

1. Project 25 Full-Rate (P25 FR) Reference System with vocoder DSP version 1.60 (software 

version 1.40e)

2. Project 25 Half-Rate (P25 HR) Reference System with vocoder DSP version 1.60 (software 

version 1.40e)

3. 25 kHz AFM reference system

4. 12.5 kHz AFM reference system

For each of the reference systems described below, the input file was a 48 kHz sampled 16-bit 

WAV file recorded as indicated in Section 2.1. The output files were the same format and were 

used to play out audio for the listeners.

 

5 This level reflects the 85 dBA maximum speech level at the measurement point specified in section 8.25 of NFPA 

1981[10], the -5 dB change in the listening environmental noise to avoid listener fatigue, an estimated +10 dB gain 

from the voice amplifier, and the -6 dB change caused by moving the measurement point from 1.5 m from the 

speech source to 3.0 m from the speech source (i.e., 85 – 5 + 10 – 6 = 84 dBA).
10

2.2.1 Project 25 Reference Systems

Figure 5 shows the configuration of the two P25 Reference Systems. The two P25 reference 

systems were the same executable code and could accept a variety of parameters as well as text 

files representing the bit error conditions on the RF channel for each bit in the signal. This 

enabled the same executable code to be used for both the P25 FR and P25 HR conditions. The 

only difference between them was calling the software with the “-fr” flag for full rate and the 

“-hr” flag for half-rate. 

To incorporate bit-errored channel conditions, the software accepted two files as input. One file 

had values of 0 or 1 to respectively represent absence or presence of a bit error. This is often 

called a BER file. The other file had numbers in the range 0.000 to 1.000 to represent log 

likelihood ratios (LLRs) for each bit. Using an LLR value closer to unity represents greater 

certainty that the corresponding bit is correct.

The BER and LLR files for the P25 FR conditions assume the underlying RF channel was a 

12.5 kHz channel operating at 9.6 kb/s with C4FM modulation. The BER and LLR files were

sieved to produce files for 7.2 kb/s that correspond to the bits on the channel that were used by 

the vocoder. The vocoder bit rate was 7.2 kb/s.

The BER and LLR files for the P25 HR conditions assumed the underlying RF channel is a 

12.5 kHz two-slot TDMA channel operating at 12 kb/s with HDQPSK modulation.6The BER 

and LLR files were sieved to produce files for 3.6 kb/s that correspond to the bits on the channel 

that were used by the vocoder. The vocoder bit rate was 3.6 kb/s.

Other parameters used default values in the software, in particular the number of soft decision 

bits (sdbits = 8) and the automatic gain control setting (off).

 

6 HDQPSK modulation is used from the fixed station to the subscriber units in P25 Phase 2. It was chosen for this 

experiment because the channel models are more mature than those for HCPM, the modulation used by the 

subscriber units in Phase 2.
11

Processed Speech and Noise

Mixed Speech & Noise .WAV file Encoder Input Filter Down- sample

Intermediate File Saved

+ Up- sample Decoder Output Filter Intermediate File Saved

bit errors 1000Hz test tone levels -28dBov -28dBov -28dBov

-28dBov 1000Hz test -28dBov -28dBov tone levels Figure 5. P25 reference systems configuration.

Characteristics associated with the P25 reference systems are listed below:

1. Input filter was a two-stage filter with high-pass and low-pass stages

a. Low pass filter has 3700 Hz passband, ≥ 70 dB stopband attenuation (≥ 4300 Hz)

b. High-pass filter was 2nd order DC blocking filter with 50 Hz cutoff

2. Downsampling from 48 kHz to 8 kHz sample rate was done with the change rate program 

defined in [16], utilizing the high-quality anti-aliasing filter

3. Vocoder was the Project 25 Dual-Rate Version 1.60 equivalent in MS-DOS executable form 

using command line flags to select full-rate (-fr) and half-rate (-hr)

4. Upsampling from 8 kHz to 48 kHz sample rate was done with the change rate program 

defined in [16], utilizing the high-quality reconstruction filter

5. Output filter had the same characteristics as input filter

2.2.2 25 kHz AFM Reference System

Figure 6 shows the configuration of the 25 kHz AFM reference system. The configuration 

included representative filters as would be used in typical analog transmitters and receivers, as 
12

well as frequency modulation, deviation limiting, and frequency demodulation. The 25 kHz 

AFM reference system passed the spec limits for audio filtering and deviation limiting given in 

the TIA-603 standard for 25 kHz analog channels [12]. While this is representative of typical 

radio designs, alternative designs are also possible.

Processed Speech and Noise

Mixed Speech & Noise .WAV file Frequency mod Band pass Pre- emphasis

+

noise fading 1000Hz test tone levels -28dBov -28dBov -28dBov

1000Hz test -28dBov -28dBov tone levels

Deviation limiter Low pass

3000 Hz pk

IF filter High pass Frequency demod De- emphasis Low pass

FM transmitter

FM receiver

CTCSS tone 500 Hz dev.

Figure 6. 25 kHz AFM reference system configuration.

The 25 kHz AFM reference system was implemented in MATLABTM code. Characteristics 

associated with the 25 kHz AFM reference system are listed below:

1. Level Setting

a. 1000 Hz tone at -28 dBov produced 60% deviation in RF modulated signal

2. Assumed equivalent of a microphone input, so guard tone notch filter not needed

3. Bandpass was a two-stage filter with high-pass and low-pass stages

a. High pass was a Type 2 Chebyshev HP (N = 5, -40 dB at 190 Hz)

b. Low pass was a Type 1 Chebyshev LP (N = 4, -0.5 dB at 3000 Hz)

4. Preemphasis was according to TIA 603 (1 pole at 5000 Hz, slope +6 dB/octave from 300 to

3000 Hz)

5. Assume CTCSS was used

6. Deviation limiter assumed a hard limiter at 5 kHz deviation
13

7. Low pass filter was a windowed FIR LPF

8. Frequency modulation responds beyond 6 kHz

9. The IF filter was a cascade of three sections. Each section was a 4 pole Butterworth design 

with nominal 3dB BW of 15.152 kHz. The effective noise bandwidth (ENBW) of the overall 

IF filter was designed to equal 13.000 kHz.

10. Frequency demodulation responded beyond IF bandwidth

11. Deemphasis was according to TIA 603 [12] (1 pole at 300 Hz, slope -6 dB/octave from 300

to 3000 Hz)

12. Low pass filter was a Butterworth (4 pole, 3000 Hz)

13. High pass filter was a Type 2 Chebyshev HP (N=5, -40 dB at 190 Hz)

14. The reference system used 48 kHz sample rate, which is more than sufficient to model the 

internal audio and IF signals.

2.2.3 12.5 kHz AFM Reference System

Figure 7 shows the configuration of the 12.5 kHz AFM reference system. The configuration 

included representative filters as would be used in typical analog transmitters and receivers, as 

well as frequency modulation, deviation limiting, and frequency demodulation. The 12.5 kHz 

AFM reference system passed the spec limits for audio filtering and deviation limiting given in 

the TIA-603 standard for 12.5 kHz analog channels. While this is representative of typical radio 

designs, alternative designs are also possible.
14

Processed Speech and Noise

Mixed Speech & Noise .WAV file Frequency mod Band pass Pre- emphasis

+

noise fading 1000Hz test tone levels -28dBov -28dBov -28dBov

1000Hz test -28dBov -28dBov tone levels

Deviation limiter Low pass

1500 Hz pk

IF filter High pass Frequency demod De- emphasis Low pass

FM transmitter

FM receiver

CTCSS tone 250 Hz dev.

Figure 7. 12.5 kHz AFM reference system configuration.

The 12.5 kHz AFM reference system was implemented in MATLABTM code. Characteristics 

associated with the 12.5 kHz AFM reference system are listed below:

1. Level Setting

a. 1000 Hz tone at -28 dBov produced 60% deviation in RF modulated signal

2. Assumed equivalent of a microphone input, so guard tone notch filter not needed

3. Bandpass was a two-stage filter with high-pass and low-pass stages

a. High pass was a Type 2 Chebyshev HP (N = 5, -40 dB at 190 Hz)

b. Low pass was a Type 1 Chebyshev LP (N = 4, -0.5 dB at 3000 Hz)

4. Preemphasis was according to TIA 603 (1 pole at 5000 Hz, slope +6 dB/octave from 300 to

3000 Hz)

5. Assume CTCSS was used

6. Deviation limiter assumed a hard limiter at 2.5 kHz deviation

7. Low pass filter was a windowed FIR LPF

8. Frequency modulation responded beyond 6 kHz
15

9. The IF filter was a cascade of 3 sections. Each section was a 4 pole Butterworth design with 

nominal 3dB BW of 9.434 kHz. The ENBW of the overall IF filter was designed to equal 

8.000 kHz.

10. Frequency demodulation responded beyond IF bandwidth

11. Deemphasis was according to TIA 603 [12] (1 pole at 300 Hz, slope -6 dB/octave from 300

to 3000 Hz)

12. Low pass filter was a Butterworth (4 pole, 3000 Hz)

13. High pass filter was a Type 2 Chebyshev HP (N=5, -40 dB at 190 Hz)

14. The reference system used 48 kHz sample rate, which is more than sufficient to model the 

internal audio and IF signals.

2.3 Acoustic Environments

The experiment tested the performance of the communication systems (and their respective 

vocoders) with background noise mixed in with the speech. The background noise was mixed 

with the speech for a specific signal-to-noise ratio. Four acoustic environments were evaluated in 

this experiment: 

1. No background noise (no mask)

2. Mask with no background noise

3. Two PASS alarms (with mask)

4. Night club noise (with no mask).

This experiment was intended to include acoustic environments that occur with talkers who are 

wearing an SCBA mask as standardized in the NFPA 1981 standard [10]. The SCBA mask 

covers the entire face, and provides a regulated air supply for the wearer. Therefore, the SCBA 

mask has a significant effect on the audio to be transmitted through the radio system. That is a 

subject of interest to this experiment. The physical configuration used to create the acoustic 

environments is described in Section 4.2.

A PASS alarm is a device commonly used by fire fighters, often while an SCBA mask is also 

used. The PASS alarm is standardized in the NFPA 1982 standard [13]. When the PASS alarm is 

activated it emits a loud, high frequency audible signal. One possible scenario for having to 

communicate in the presence of PASS noise is for a rescue team to talk on radios while 

recovering a downed two-person team. This scenario involved two PASS alarms from a single 

manufacturer mixed together for the background noise. The PASS alarm signals were mixed to 

provide the most consistent noise level possible. In the case of PASS 1, the second signal was 

offset by 43% of the alarm cycle duration. In the case of PASS 2, the second signal was offset by 

45% of the alarm cycle duration.
16

The noise conditions represented acoustic noise environments from which a user might be 

transmitting and that are commonly known to cause difficulty with communications. The SNRs 

were chosen to approximate sound level conditions in the application environment.

The power spectral densities (PSDs) and spectrograms of the three background noises are given 

in Figures 8 through 10. The PSDs show relative power versus frequency. The spectrograms 

show power (by color) as a function of time (on the horizontal axis) and frequency (on the 

vertical axis). Together the PSDs and spectrograms provide an indication that the noise 

environments cover a wide range of frequency characteristics, impulse characteristics, and 

amplitudes. The mask acoustic transfer characteristic is given in Figure 11. There is notable 

signal loss in the 300–500 Hz range and the 1500–3000 Hz range. This is significant because 

these ranges of frequencies are important to intelligibility.

The PSDs in Figures 8 through 10 and the acoustic path loss in Figure 11 were computed using 

MATLABTM mathematical analysis software. The computation method (Welch) [14], number of 

elements in the transform (Nfft), and the size and shape of the computational window (Hamming 

[14], 1024) were parameters provided to the MATLABTM algorithms.
17

Figure 8. PSD plot and spectrogram of two PASS alarms from manufacturer 1 sounding.

Spectrogram has x-axis in seconds and y-axis in Hz.
18

Figure 9. PSD plot and spectrogram of two PASS alarms from manufacturer 2 sounding.

Spectrogram has x-axis in seconds and y-axis in Hz.
19

Figure 10. PSD plot and spectrogram of night club noise. Spectrogram has x-axis in seconds and 

y-axis in Hz.
20

Figure 11. Acoustic path loss through SCBA mask.

2.4 Channel Impairments

In addition to emulating a clean RF channel, this experiment also used emulations of static 

channel degradations. Table 2 shows degraded static RF channel conditions applied to all four 

noise conditions. 

Table 2. Degraded static RF channel conditions.

System RF Level IF Noise Bandwidth CNR

25 kHz Analog -117 dBm 13.0 kHz 6.9 dB

12.5 kHz Analog -117 dBm 8.0 kHz 9.0 dB

P25 FR -117 dBm 5.8 kHz 10.4 dB

P25 HR -117 dBm 6.0 kHz 10.2 dB
21

2.5 Test Conditions

The noise environments, mask presence, and RF channel conditions used in the experiment are 

provided in the first three columns of Table 3. The noise environment, mask, and RF channel 

scenarios combine with the four reference systems to provide the list of conditions. Table 3

shows which noise environments and channel conditions were used with which reference 

systems. Appendix A contains a full list of conditions for the experiment.

Table 3. Combinations of scenarios and communication systems.

Noise Environment Channel

SNR 

(dB)

25 kHz 

AFM P25 FR

12.5 kHz 

AFM P25 HR

No noise, no mask Clean X X X X

Mask (vox port) with no noise Clean X X X X

Mask (int microphone) with no 

noise Clean X X X X

PASS Alarm 1, mask (vox port) Clean -2

(7)

X X X X

PASS Alarm 2, mask (vox port) Clean -2

(7)

X X X X

PASS Alarm 1, mask (int 

microphone)

Clean -2 X X X X

PASS Alarm 2, mask (int 

microphone)

Clean -2 X X X X

Night club noise, no mask Clean 5 X X X X

No noise, no mask Degraded X X X X

Mask (vox port) with no noise Degraded X X X X

Mask (int microphone) with no 

noise Degraded X X X X

PASS Alarm 1, mask (vox port) Degraded -2

(7)

X X X X

PASS Alarm 1, mask (int 

microphone)

Degraded -2 X X X X

Night club noise, no mask Degraded 5 X X X X

 

7 This does not include attenuation of the signal due to the mask (approximately 9 dB).
22

3 SPEECH DATA BASES

The speech source material used for the test consisted of spoken word lists from the Modified 

Rhyme Test (MRT) described in [9]. Source material used for the tests had a quiet acoustic 

background. In the experiment, acoustic background noise was added to the source material. The 

MRT was used for evaluation. These tests placed several conditions on the speech data base:

1. Large numbers of words 

2. Equalized presentation levels 

3. Sentences of equivalent content and structure.

The speech source material for the test was recorded at ITS.8This material was the MRT word 

list defined in [9] using the carrier sentence, “Please select the word ...” More specific 

information about the recording process for this source material is given in Appendix C.

3.1 Speech Data Base Properties

Fifty lists of six words from the data base were processed with each test condition. Each coding 

mechanism processed the same speech material under the same operating conditions. Sufficient 

source material was used to prevent listeners from being presented with repeat material. To

reduce order bias, the presentation order of the material to the listeners was randomized by talker 

and condition. Within a given talker/condition, samples were also randomized, but were 

presented as a block to the listener.

Material for two male and two female talkers9 was used as described in Error! Reference 

source not found.. Each talker spoke each of 300 MRT sentences. This resulted in 1200 

sentences for each of the fourteen scenarios. A total of 16800 sentences were then processed with 

each of four reference systems, resulting in 67200 files. System 1 is 25 kHz AFM, System 2 is 

12.5 kHz AFM, System 3 is P25 FR [Enhanced Dual-Rate Software Version 1.40e], System 4 is 

P25 HR [Enhanced Dual-Rate Software Version 1.40e].

 

8

ITS and PSCR have made speech material from this test available on the internet to enable continued research in 

this area. The material can be found here: http://www.pscr.gov/projects/audio_quality/mrt_library/mrt_library1.php. 

9

[10] specifies an unbalanced talker pool of four male and one female talkers. To provide more gender-balanced 

results that could have wider applicability, a balanced talker pool of two male and two female talkers was used.
23

Table 4. Source audio arrangement of sentences, test conditions, and talkers.

Talker (2 male, 2 female) 1 2 3 4

Talker Gender M M F F

Clean RF Channel

No noise, no mask 300 300 300 300 1200

No noise, mask (vox port) 300 300 300 300 1200

PASS Alarm 1 (-2 dB SNR) 10, Mask (vox port) 300 300 300 300 1200

PASS Alarm 2 (-2 dB SNR), Mask (vox port) 300 300 300 300 1200

Night club noise (5 dB SNR), no mask 300 300 300 300 1200

No noise, mask (int microphone) 300 300 300 300 1200

PASS Alarm 1 (-2 dB SNR), Mask (int microphone) 300 300 300 300 1200

PASS Alarm 2 (-2 dB SNR), Mask (int microphone) 300 300 300 300 1200

Static Degraded RF Channel

No noise, no mask 300 300 300 300 1200

No noise, mask (vox port) 300 300 300 300 1200

No noise, mask (int microphone) 300 300 300 300 1200

PASS Alarm 1 (-2 dB SNR),

10 Mask (vox port) 300 300 300 300 1200

PASS Alarm 1 (-2 dB SNR), Mask (int microphone) 300 300 300 300 1200

Night club noise (5 dB SNR), no mask 300 300 300 300 1200

16800

The speech recordings were made in controlled acoustic noise environments. The testing 

included two noise-free conditions, three acoustic noise environments, and two channel 

conditions. For the experiment, there were two male talkers and two female talkers. 

The speech material was equalized across all talkers for presentation level. The material was 

recorded in WAV files in full audio bandwidth and dynamic range with no additional processing. 

The speech material processed through the reference systems was evaluated by a total of 52 

listeners. No more than five listeners were presented the same playback order randomization, no 

listener was presented repeated word lists, and the combinations of word list, communication 

technology, and condition were balanced across the total listener crew.

3.2 Speech Data Base Levels

An important attribute of the speech data base, especially for the inputs to the reference systems, 

was the average power level of the speech material. The nominal power level for speech 

followed the recommendation defined in [1] and excerpted below. This recommendation was 

followed for each sentence.

 

10 This does not include attenuation of the signal due to the mask (approximately 9 dB).
24

“It is recommended that the analog input gain be set such that the RMS speech level 

under nominal input conditions is 25 dB below the saturation point of the A-to-D 

converter. This level (-22 dBm0) is designed to provide sufficient margin to prevent the 

peaks of the speech waveform from being clipped by the A-to-D converter.”

All input, playback and reference systems were tuned to maintain the -22 dBm0 (-28 dBov) 

±0.5 dB active speech level in the digital files throughout the processing and playback. This 

ensured that the speech was always delivered to the listeners as if the talker were speaking at a 

constant volume.

A further specification is given for the measurement method to determine the average speech 

power level. The varying nature of speech signal amplitude presents a difficulty for accurately 

measuring the level using an arbitrary method. The tests reported here used the ITU-T 

Recommendation P.56 [17] method B to accurately measure the active speech level.

3.3 Signal and Acoustic Noise Levels

In order to construct the appropriate signal-to-noise levels for input to the reference systems, 

careful attention to the noise signal filtering and level adjustment of the signals was necessary. 

The definition for signal-to-noise ratio in dB for this test is:

 (1)

Active speech level was computed according to [17]. The short-term power function of each 

noise was assumed to be much more stationary than speech, relieving the need for an 

activity/threshold detector. The computation for the noise levels followed an RMS algorithm that 

was scaled to the overload point ( ) so that dB values are negative or zero:

 

 

 √

∑ 

 

 

 

(2)

where is the 

 

sample of the noise signal with length samples. The noise samples were 

scaled by the appropriate factor to obtain the target noise level for that specific test condition,

summed together acoustically as described in Section 4.2, and then recorded as WAV files to 

create a noise condition file for reference system input.
25

4 PRODUCTION OF RECORDED FILES

This section describes the test procedure that was followed in conducting the tests of the 

reference systems. This procedure was designed to assist interested parties to reproduce the 

speech files for later scoring by a listening laboratory. 

4.1 Test Elements

The test used the following elements:

1. Head and torso simulators [18] [19]

2. NC-35 sound attenuated chamber

3. Representative SCBA masks

4. Ability to produce environmental noise at appropriate level within the NC-35 chamber

5. Reference systems as described in Section 2

6. Recording and playback hardware and software

4.2 Test Signal Preparation

Generating the test material was a matter of processing 300 sentences for each of the four talkers 

through the permutations of acoustic and channel conditions specified for the experiment. This 

was done in two stages. The first stage was the recording procedure described below combined 

with the recording system described in Section 2.1. The second stage was processing that 

recorded material through the four reference systems with both clean and degraded channel 

conditions. The output files from the reference systems were then played to the listeners as 

indicated in Section 5.3.

For the conditions with background noise, it was important that the background noise be active 

in the first sample of the recorded file to avoid potential false training of features of the codec 

(i.e., mislead the coder into thinking it is starting in a quiet environment), which can lead to 

longer-than-normal training times once the noise starts.

Figure 12 shows the physical configuration for those conditions without a mask, and Figure 13

shows the physical configuration for those conditions with a mask and using an external 

microphone. The indicated microphone was used to record speech from the HATS acoustically 

mixed with background noise generated through the loudspeakers. Figure 14 shows the physical 

configuration for those conditions using a mask with an internal microphone.

For those conditions without a mask, the recording microphone was positioned 5 cm (2 in.) 

directly in front of the lip reference point (LRP). For those conditions using the voice port on a 

mask, the recording microphone was positioned against the voice transmission port on the mask. 
26

For those conditions using a mask with an internal microphone, the manufacturer-supplied 

internal microphone (positioned inside the breathing cone of the mask) was used.

Because both of the background noises used in this experiment were multipoint noise sources, all 

five speakers were used to generate the noise.

For the purposes of this experiment, the artificial mouth of the HATS was equalized to flat ±1 dB 

in the band of 160 Hz to 10 kHz. Speech was played through the HATS at a level of 100 dBC at 

the LRP (measured without a mask in place). This was consistent with measurements made on 

behalf of TIA of users talking in a loud noise environment. For those conditions with background 

noise, the noise SPL was also measured in dBc at the LRP without a mask in place. These noise 

and speech levels were used to achieve the specified SNR and ensure that the sound levels were 

consistent in the chamber and the effects of the masks and microphones on the reference systems 

could be evaluated.

2'-6"

2'-0"

Figure 12. Physical configuration showing acoustic anechoic chamber, speakers, microphone,

and HATS for non-mask conditions.
27

2'-6"

2'-0"

Figure 13. Physical configuration showing acoustic anechoic chamber, speakers, SCBA mask, 

microphone, and HATS for conditions with SCBA mask and external microphone.

2'-6"

2'-0"

Figure 14. Physical configuration showing acoustic anechoic chamber, speakers, SCBA mask 

with internal microphone, and HATS for conditions with SCBA mask and internal microphone.
28

5 SUBJECTIVE EVALUATION OF INTELLIGIBILITY

The subjective evaluation of intelligibility using the MRT is specified in [9] and [10] and 

described in [6]. 

Subjective testing involves the use of a number of listeners who attempt to interpret the words 

spoken through the reference system. The processed speech samples were obtained as described 

in Section 4. Because intelligibility is subject to individual abilities to interpret spoken words, 

reliable results are obtained when a number of listeners are used. For this assessment, a total of 

52 listeners were used. To mitigate the effects of the order of presentation of the speech samples 

to the listeners, a different presentation order was used with each listener.

5.1 Stimulus File Preparation

After processing the input material with all coding mechanisms for the experiment, the proper 

sentences needed to be assembled. This material was then randomized in its order of presentation 

to the listeners. Each group of listeners listened to a different subset of 2400 sentences that were 

drawn from the recordings made using the input, output, and reference systems described in 

Section 2.

5.2 MRT Evaluation Laboratory

The ITS laboratory used to perform the tests was configured to conform as closely as possible to 

the applicable sections of ITU-T Recommendation P.800 [15],

11 as described in Appendix D. 

ITS, acting as the Evaluation Laboratory under the Recommendation, both conducted the tests 

(as described in Section 5.3) and delivered the results of the experiment (as described in 

Section 5.4).

Prior to the start of the test the listeners participated in a practice session. During this practice 

session they were presented with 112 practice sentences, two from each condition, which they 

scored. The practice sentences consisted of a block of material as described in Section 3.1, but 

taken from the larger corpus of speech material, excluding the samples for the experimental 

sessions. After the practice session the listeners were asked if they understood what they were 

supposed to do. If there were any questions they were answered at that time. After that the 

formal test began. The purpose of the practice session was to: (a) expose the listener to the range 

of audio quality of the test, and (b) to see if they understood what they were supposed to be 

doing. This was in accordance with [15], clause B.4.6, Instructions to subjects. 

 

11 Listening tests are described in Annex B of ITU Recommendation P.800. Because P.800 was designed for 

telephony applications and not public safety land mobile radio applications, certain adaptations are made to 

accommodate other standards that are an integral part of this test. Specifically, the listening environment noise and 

noise levels are based on NFPA 1981-2007 (specifying pink noise), rather than those specified in P.800 (Hoth 

noise). Also, TIA TR-8 prefers an active speech level of -28 dBov in comparison to the -26 dBov recommended in 

P.800. Finally, the MRT intelligibility test from ANSI S3.2 is the method used instead of the quality methods 

described in P.800.
29

The experimental results are presented, while the data from the practice sessions were discarded. 

The results from the listening test were then permuted to undo the randomization described in 

Section 5.1, and then the results reported as described in Section 5.4.

5.3 Intelligibility Testing

The subjective evaluation consisted of one experiment to determine effects of background noise 

and channel degradation.

The experiment followed the Modified Rhyme Test (MRT) method [9] to assess intelligibility. In 

the MRT, each listener listened to a sentence asking them to select a word from a prescribed list. 

The listeners’ ability to select the correct word was averaged across listeners and produced a 

percentage of intelligibility score.

The MRT consisted of the conditions shown in Table 3.

Presentation of speech material was made via high fidelity near-field monitor speaker at a 

distance of 1.5 m from the listener. The playback system was calibrated to deliver an average 

speech listening level of 84 dBA when measured at the listening position. The equivalent 

acoustic noise level of the playback system did not exceed 35 dBA. 

Listeners were seated in a room, with an ambient pink noise level of 65 dBA as defined in 

subclauses 8.10.4.11 through 8.10.4.15 of [10]. The exact configuration of the room and 

characterization of the noise is shown in Appendix D.

Notwithstanding that the listeners were practitioners, they were naïve with respect to 

communication technology issues; that is, they were not considered experts in telephone design, 

digital voice encoding algorithms, and so on. The sample included adults of mixed sex, age, and 

practitioner discipline. Persons were given a hearing screening and results were used only from 

those subjects who had audiometrically normal hearing as defined in subclause 8.10.4.3 of [10]. 

The test was conducted as described in subclauses 8.10.5.1 and 8.10.5.2 of [10], with the 

exceptions that the carrier sentence was “Please select the word [list word]” and that listeners 

selected the word on a touch screen.

The administration of the experiment was as follows: The processed speech was presented to a 

panel of 52 listeners. Thirty-two of fifty-two listeners heard 14 lists of 150 sentences (or 2100 

sentences) for a grand total of sentences. However, it was discovered 

that due to faulty recording equipment, all sentences for 20 conditions ( 

sentences) were recorded improperly. The sentences were re-recorded and retested. Twenty of 

fifty-two listeners heard 10 lists of 150 sentences (or 1500 sentences) for a grand total of 

 sentences. There was some overlap with the initial test (6000 sentences 

or 5 conditions) which enabled a small amount of repeatability testing.

Before starting the experiment, the subjects were given the following instructions. The 

instructions can be modified to allow for variations in laboratory data-gathering apparatus.
30

Instructions for Subjects

[Before Training Session]

Welcome and thank you for coming.

This experiment is seven sessions of approximately 20 minutes each. You will be able 

to take a break after each session, and we will have you take at least a five-minute 

break after every second session. This experiment involves no risk or discomfort, and 

you are free to leave the experiment at any time for any reason, it will not be a problem 

for us. If you have any questions about the experiment, please feel free to ask them 

before the experiment starts. Your responses will be kept confidential, and will only be 

reported as statistics for this experiment. 

This experiment uses the speakers in the room, so you will not be able to adjust the 

volume. 

The purpose of this experiment is to gather intelligibility information on systems that 

might be used for communications service between separate locations. You will be 

hearing a number of samples of speech reproduced in the speaker. Each sample will 

consist of the sentence “Please select the word X” spoken by male or female talkers.

Please listen to the sentence, and then select the requested word from the list on the 

PDA by tapping quickly on the screen with your fingertip. You may hear background 

noise in some of the samples. Please do your best to pick the requested word.

Any questions?

[Between Training Session and Session 1]

Any problems during the training session?

We will now do the first session of 300 samples. Any questions before we begin?

5.4 Analysis and Reporting of Results

The results are reported in a series of tables in Section 6. The analysis and reporting is outlined 

in sections 8.10.5 and 8.10.6 of [10]. Averages were computed using the adjusted method 

recommended for closed set tests as described in section 10.2 of [9]. An analysis of variance 

(ANOVA) was computed to enable comparisons between the implementations. A more detailed 

description of the analysis follows.

ANOVA and a multiple comparison test can assist in the determination of whether there is a 

significant variation between the speech outputs of the four reference systems (and their 
31

respective vocoder technologies), and if so, which is better. A common multiple comparison test 

used in previous tests is the Tukey pairwise comparison test.

The data under analysis with ANOVA and Tukey consisted of adjusted average intelligibility 

scores, RA (described in section 10.2 of [9]), collected for 4 communication technologies, 4 

talkers (each speaking 300 sentences), 14 acoustic noise/channel conditions (for a grand total of 

 sentences), and a total of 52 listeners. The results presented in this 

report represent data collected using files from the 36 correctly-recorded conditions during the 

initial test and the 20 correctly-recorded conditions during the subsequent test. Together, both 

tests fully cover the individual conditions contained in the test. 

ANOVA compared the variance of the overall sample population with the variance within each 

sub-population, and if they exceeded a value given by the Fisher F-distribution, then the null 

hypothesis was false. In this case, the null hypothesis was that communication technologies do 

not make a difference in intelligibility.

The hypothesis under test, H, was that the communication system (in particular the vocoder 

implemented by that communication technology) affected the intelligibility, RA, as measured by 

the MRT. The hypothesis can be tested for 13 of the 14 acoustic noise/channel conditions. If the 

hypothesis is true, it is also desired to know which communication technology is better.

For this test, α = 0.01 was used for the F-distribution in the ANOVA for the given degrees of 

freedom, df, and sample size, n. These parameters were programmed into a statistical analysis 

package (Minitab®) for computation of the ANOVA and Tukey pairwise comparison. Analysis 

results are summarized in Section 6, and individual condition analyses are presented in Appendix 

A.
32

6 TEST RESULTS

This section describes the results of the experiment. One-way ANOVA results were computed 

using a statistical analysis package. Input to the ANOVA consisted of six adjusted intelligibility 

scores (RA) per talker/noise environment/reference system. Each of those adjusted intelligibility 

scores were based on the intelligibility of 50 samples for that particular talker/noise 

environment/reference system combination that were presented to that particular listener.

The RA listener scores for the experiment are tabulated in Appendix B. The analysis of the scores 

for the experiment is given in Section 6.1. The analysis given in Section 6.1 includes the analysis 

of the RA scores resulting from:

 the evaluation of the 36 correctly-recorded conditions during the initial test

 the evaluation of the 20 correctly-recorded conditions during the subsequent test 

The subsequent test included 6 conditions that overlapped with 6 of the 36 correctly-recorded 

conditions utilized during the initial test. These overlapping results were tested for statistical 

similarity and were found to be statistically similar. This says that the test methodology creates 

repeatable results, and therefore the RA scores from the subsequent test are included with the RA

scores from the initial test.

In Table 3 a total of 56 conditions are enumerated for testing (Error! Reference source not found.

details each condition more explicitly). However, an error in the file creation process resulted in 

an erroneous representation of conditions 13 and 27 (mask (internal microphone) with PASS 1 (-

2 dB SNR) background noise and statically degraded channel for both 25 kHz and 12.5 kHz 

AFM). This error was not discovered until after both the initial and subsequent tests were 

completed. Therefore the results that follow only include the 54 remaining, valid conditions. In 

light of this, statistical significance tests are not available for Scenario 13.

6.1 Experiment Results by Scenario

Experimental results are presented below for each noise environment (scenario) in the test. For 

each scenario, analysis of the one-way ANOVA (p = 0.01) is presented. If a statistically 

significant difference was detected by the ANOVA, a Tukey pairwise comparison analysis (α = 

0.01) follows to reveal which elements are different, from a statistical standpoint. Finally, the 

results for the scenario are presented in graphical form.

6.1.1 Results for Scenario 1: “No Noise, No Mask, Clean RF Channel”

Scenario 1 consisted of the ideal communications case with no background noise, no SCBA 

mask, and a clean RF channel. The listening environment was as described in Section 5.3. Table 

5 contains the intelligibility scores, ANOVA, and Tukey results for this environment. The 

detailed Minitab report is found in Appendix A.
33

Figure 15 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that only the 12.5 kHz AFM was 

different from the P25 HR system. All of the other five paired comparisons showed statistically 

similar scores.

Table 5. Intelligibility scores, ANOVA results, and Tukey results for Scenario 1.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.896 0.907 0.866 0.843

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.06082 0.02027 5.83 0.001 YES

Error 92 0.32011 0.00348

Total 95 0.38093

S = 0.05899 R-Sq = 15.97% R-Sq(adj) = 13.22%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.896 0.04835 NO NO NO

2 - 12.5 kHz AFM 0.907 0.0497 YES NO

3 - P25 Full Rate 0.866 0.06483 NO

4 - P25 Half Rate 0.843 0.07005
34

Figure 15. Intelligibility scores (RA) for Scenario 1.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 1
35

6.1.2 Results for Scenario 2: “No Noise, Mask (Vox Port), Clean RF Channel”

Scenario 2 consisted of no background noise, SCBA mask with vox port, and a clean RF 

channel. The listening environment was as described in Section 5.3. Table 6 contains the 

intelligibility scores, ANOVA, and Tukey results for this environment. The detailed Minitab 

report is found in Appendix A.

Figure 16 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that only the 12.5 kHz AFM was 

different from the P25 HR system. All of the other five paired comparisons showed statistically 

similar scores.

Table 6. Intelligibility scores, ANOVA results, and Tukey results for Scenario 2.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.754 0.771 0.692 0.672

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.16379 0.0546 6.55 0.000 YES

Error 92 0.76673 0.00833

Total 95 0.93052

S = 0.09129 R-Sq = 17.60% R-Sq(adj) = 14.92%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.754 0.0994 NO NO NO

2 - 12.5 kHz AFM 0.771 0.06897 YES NO

3 - P25 Full Rate 0.692 0.1005 NO

4 - P25 Half Rate 0.672 0.09273
36

Figure 16. Intelligibility scores (RA) for Scenario 2.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 2
37

6.1.3 Results for Scenario 3: “No Noise, Mask (Int Mic), Clean RF Channel”

Scenario 3 consisted of no background noise, SCBA mask with internal microphone, and a clean 

RF channel. The listening environment was as described in Section 5.3. Table 7 contains the 

intelligibility scores, ANOVA, and Tukey results for this environment. The detailed Minitab 

report is found in Appendix A.

Figure 17 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The 25 kHz AFM and 12.5 kHz AFM are in one group with the higher 

scores, and the P25 FR and P25 HR are in the second group with the lower score.

Table 7. Intelligibility scores, ANOVA results, and Tukey results for Scenario 3.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.840 0.831 0.712 0.723

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.3366 0.1122 19.18 0.000 YES

Error 92 0.53813 0.00585

Total 95 0.87473

S = 0.05926 R-Sq = 14.82% R-Sq(adj) = 11.71%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.84 0.09698 YES YES NO

2 - 12.5 kHz AFM 0.831 0.07047 YES YES

3 - P25 Full Rate 0.712 0.1005 NO

4 - P25 Half Rate 0.723 0.09066
38

Figure 17. Intelligibility scores (RA) for Scenario 3.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 3
39

6.1.4 Results for Scenario 4: “PASS 1, Mask (Vox Port), Clean RF Channel”

Scenario 4 consisted of PASS alarm 1 background noise, SCBA mask with vox port, and a clean 

RF channel. The listening environment was as described in Section 5.3. Table 8 contains the 

intelligibility scores, ANOVA, and Tukey results for this environment. The detailed Minitab 

report is found in Appendix A.

Figure 18 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two

groups. The 25 kHz AFM and 12.5 kHz AFM scores are in one group. The other group includes 

12.5 kHz AFM along with P25 FR and P25 HR scores. The 12.5 kHz AFM score overlaps both 

groups. The 25 kHz analog system had higher intelligibility than the two P25 systems.

Table 8. Intelligibility scores, ANOVA results, and Tukey results for Scenario 4.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.606 0.586 0.499 0.508

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.21112 0.07037 7.06 0.000 YES

Error 92 0.91754 0.00997

Total 95 1.12866

S = 0.09987 R-Sq = 18.71% R-Sq(adj) = 16.05%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.606 0.09413 YES YES NO

2 - 12.5 kHz AFM 0.586 0.11218 NO NO

3 - P25 Full Rate 0.499 0.08353 NO

4 - P25 Half Rate 0.508 0.1071
40

Figure 18. Intelligibility scores (RA) for Scenario 4.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 4
41

6.1.5 Results for Scenario 5: “PASS 2, Mask (Vox Port), Clean RF Channel”

Scenario 5 consisted of PASS alarm 2 background noise, SCBA mask with vox port, and a clean 

RF channel. The listening environment was as described in Section 5.3. Table 9 contains the 

intelligibility scores and ANOVA results for this environment. The detailed Minitab report is 

found in Appendix A.

Figure 19 contains the bar chart of results for this environment. The ANOVA did not reveal a 

statistically significant difference, thus a Tukey test was not performed.

Table 9. Intelligibility scores and ANOVA results for Scenario 5.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.554 0.550 0.485 0.481

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.1146 0.0382 3.3 0.024 NO

Error 92 1.0644 0.0116

Total 95 1.179

S = 0.1076 R-Sq = 9.72% R-Sq(adj) = 6.78%
42

Figure 19. Intelligibility scores (RA) for Scenario 5.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 5
43

6.1.6 Results for Scenario 6: “PASS 1, Mask (Int Mic), Clean RF Channel”

Scenario 6 consisted of PASS alarm 1 background noise, SCBA mask with internal microphone, 

and a clean RF channel. The listening environment was as described in Section 5.3. Table 10

contains the intelligibility scores, ANOVA, and Tukey results for this environment. The detailed 

Minitab report is found in Appendix A.

Figure 20 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The 25 kHz AFM and 12.5 kHz AFM are in one group with the higher 

scores, and the P25 FR and P25 HR are in the second group with the lower scores.

Table 10. Intelligibility scores, ANOVA results, and Tukey results for Scenario 6.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.800 0.822 0.707 0.710

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.25807 0.08602 16.78 0.000 YES

Error 92 0.47172 0.00513

Total 95 0.72979

S = 0.1019 R-Sq = 9.02% R-Sq(adj) = 5.81%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.8 0.07411 YES YES NO

2 - 12.5 kHz AFM 0.822 0.06405 YES YES

3 - P25 Full Rate 0.707 0.0788 NO

4 - P25 Half Rate 0.71 0.06859
44

Figure 20. Intelligibility scores (RA) for Scenario 6.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 6
45

6.1.7 Results for Scenario 7: “PASS 2, Mask (Int Mic), Clean RF Channel”

Scenario 7 consisted of PASS alarm 2 background noise, SCBA mask with internal microphone, 

and a clean RF channel. The listening environment was as described in Section 5.3. Table 11

contains the intelligibility scores, ANOVA, and Tukey results for this environment. The detailed 

Minitab report is found in Appendix A.

Figure 21 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The 25 kHz AFM and 12.5 kHz AFM are in one group with the higher 

scores, and the P25 FR and P25 HR are in the second group with the lower scores.

Table 11. Intelligibility scores, ANOVA results, and Tukey results for Scenario 7.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.817 0.817 0.715 0.711

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.25978 0.08659 17.23 0.000 YES

Error 92 0.46243 0.00503

Total 95 0.72221

S = 0.08876 R-Sq = 34.95% R-Sq(adj) = 32.82%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.817 0.08777 YES YES NO

2 - 12.5 kHz AFM 0.817 0.07747 YES YES

3 - P25 Full Rate 0.715 0.05496 NO

4 - P25 Half Rate 0.711 0.05814
46

Figure 21. Intelligibility scores (RA) for Scenario 7.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 7
47

6.1.8 Results for Scenario 8: “Night Club Noise, No Mask, Clean RF Channel”

Scenario 8 consisted of night club noise, no mask, and a clean RF channel. The listening 

environment was as described in Section 5.3. Table 12 contains the intelligibility scores, 

ANOVA, and Tukey results for this environment. The detailed Minitab report is found in 

Appendix A.

Figure 22 contains the bar chart of results for this environment The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The 25 kHz AFM and 12.5 kHz AFM are in one group with the higher 

scores, and the P25 FR and P25 HR are in the second group with the lower scores.

Table 12. Intelligibility scores, ANOVA results, and Tukey results for Scenario 8.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.683 0.656 0.494 0.503

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.7115 0.2372 16.2 0.000 YES

Error 92 1.3467 0.0146

Total 95 2.0582

S = 0.1210 R-Sq = 34.57% R-Sq(adj) = 32.43%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.683 0.1148 YES YES NO

2 - 12.5 kHz AFM 0.656 0.0992 YES YES

3 - P25 Full Rate 0.494 0.1454 NO

4 - P25 Half Rate 0.503 0.12
48

Figure 22. Intelligibility scores (RA) for Scenario 8.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 8
49

6.1.9 Results for Scenario 9: “No Noise, No Mask, Degraded RF Channel”

Scenario 9 consisted of no background noise, no mask, and a statically degraded RF channel. 

The listening environment was as described in Section 5.3. Table 13 contains the intelligibility 

scores, ANOVA, and Tukey results for this environment. The detailed Minitab report is found in 

Appendix A.

Figure 23 contains the bar chart of results for this experiment. The ANOVA results show a 

statistically significant difference in the scores. The Tukey comparison test shows that the scores 

fall into three groups with no overlap. The P25 FR and P25 HR scores are in one group with the 

highest scores. The 25 kHz AFM is in the second group with a lower score. The 12.5 kHz AFM 

is in the third group with the lowest score.

Table 13. Intelligibility scores, ANOVA results, and Tukey results for Scenario 9.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.701 0.590 0.876 0.852

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 1.30058 0.43353 57.49 0.000 YES

Error 92 0.69377 0.00754

Total 95 1.99435

S = 0.08684 R-Sq = 65.21% R-Sq(adj) = 64.08%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.701 0.08979 YES YES YES

2 - 12.5 kHz AFM 0.59 0.12009 YES YES

3 - P25 Full Rate 0.876 0.05233 NO

4 - P25 Half Rate 0.852 0.0703
50

Figure 23. Intelligibility scores (RA) for Scenario 9.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 9
51

6.1.10 Results for Scenario 10: “No Noise, Mask (Vox Port), Degraded RF Channel”

Scenario 10 consisted of no background noise, SCBA mask with vox port, and a statically 

degraded RF channel. The listening environment was as described in Section 5.3. Table 14

contains the intelligibility scores, ANOVA, and Tukey results for this environment. The detailed 

Minitab report is found in Appendix A.

Figure 24 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The P25 FR and P25 HR are in one group with the higher scores, and the 

25 kHz AFM and 12.5 kHz AFM are in the second group with the lower score.

Table 14. Intelligibility scores, ANOVA results, and Tukey results for Scenario 10.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.484 0.409 0.694 0.679

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 1.4526 0.4842 48.83 0.000 YES

Error 92 0.91224 0.00992

Total 95 2.36484

S = 0.09958 R-Sq = 61.42% R-Sq(adj) = 60.17%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.484 0.11948 YES YES NO

2 - 12.5 kHz AFM 0.409 0.11319 YES YES

3 - P25 Full Rate 0.694 0.07952 NO

4 - P25 Half Rate 0.679 0.07907
52

Figure 24. Intelligibility scores (RA) for Scenario 10.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 10
53

6.1.11 Results for Scenario 11: “No Noise, Mask (Int Mic), Degraded RF Channel”

Scenario 11 consisted of no background noise, SCBA mask with internal microphone, and a 

statically degraded RF channel. The listening environment was as described in Section 5.3. Table 

15 contains the intelligibility scores, ANOVA, and Tukey results for this environment. The 

detailed Minitab report is found in Appendix A.

Figure 25 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The P25 FR and P25 HR are in one group with the higher scores, and the 

25 kHz AFM and 12.5 kHz AFM are in the second group with the lower scores.

Table 15. Intelligibility scores, ANOVA results, and Tukey results for Scenario 11.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.614 0.542 0.728 0.730

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.60948 0.20316 24052 0.000 YES

Error 92 0.76234 0.00829

Total 95 1.37182

S = 0.08371 R-Sq = 66.26% R-Sq(adj) = 65.05%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.614 0.0944 YES YES NO

2 - 12.5 kHz AFM 0.542 0.10778 YES YES

3 - P25 Full Rate 0.728 0.08716 NO

4 - P25 Half Rate 0.73 0.07086
54

Figure 25. Intelligibility scores (RA) for Scenario 11.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 11
55

6.1.12 Results for Scenario 12: “PASS 1, Mask (Vox Port), Degraded RF Channel”

Scenario 12 consisted of PASS alarm 1 background noise, SCBA mask with vox port, and a 

statically degraded RF channel. The listening environment was as described in Section 5.3. Table 

16 contains the intelligibility scores, ANOVA, and Tukey results for this environment. The 

detailed Minitab report is found in Appendix A.

Figure 26 contains the bar chart of results for this environment. The ANOVA test indicates that 

some RA scores are different. The Tukey comparison test shows that the scores fall into two 

groups with no overlap. The P25 FR and P25 HR are in one group with the higher scores, and the 

25 kHz AFM and 12.5 kHz AFM are in the second group with the lower score.

Table 16. Intelligibility scores, ANOVA results, and Tukey results for Scenario 12.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.339 0.328 0.488 0.498

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.61322 0.20441 20.52 0.000 YES

Error 92 0.91649 0.00996

Total 95 1.52971

S = 0.09981 R-Sq = 40.09% R-Sq(adj) = 38.13%

Tukey Multiple Comparison Results “YES” means significance

System Mean StDev 4 3 2 1

1 - 25 kHz AFM 0.339 0.08979 YES YES NO

2 - 12.5 kHz AFM 0.328 0.09416 YES YES

3 - P25 Full Rate 0.488 0.11205 NO

4 - P25 Half Rate 0.498 0.1018
56

Figure 26. Intelligibility scores (RA) for Scenario 12.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 12
57

6.1.13 Results for Scenario 13: “PASS 1, Mask (Int Mic), Degraded RF Channel”

Scenario 13 consisted of PASS alarm 1 background noise, SCBA mask with internal 

microphone, and a statically degraded RF channel. The listening environment was as described 

in Section 5.3. Table 17 contains the intelligibility scores for this environment. Due to an error in 

the file creation process, the intelligibility scores for 25 kHz and 12.5 kHz AFM are invalid and 

not reported. No further statistical analysis is available.

Figure 27 contains the bar chart of results for this environment. 

Table 17. Intelligibility scores for Scenario 13.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

--- --- 0.718 0.711

Figure 27. Intelligibility scores (RA) for Scenario 13.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 13
58

6.1.14 Results for Scenario 14: “Night Club, No Mask, Degraded RF Channel”

Scenario 14 consisted of night club background noise, no SCBA mask, and a statically degraded 

RF channel. The listening environment was as described in Section 5.3. Table 18 contains the 

intelligibility scores and ANOVA results for this environment. The detailed Minitab report is 

found in Appendix A.

Figure 28 contains the bar chart of results for this environment. The ANOVA did not reveal a 

statistically significant difference, thus a Tukey test was not performed. 

Table 18. Intelligibility scores and ANOVA results results for Scenario 14.

Intelligibility Scores (RA)

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

0.542 0.470 0.530 0.463

One-way ANOVA: RA versus System 

Source DF SS MS F P Significant?

Condition 3 0.1182 0.0394 2.51 0.064 NO

Error 92 1.4464 0.0157

Total 95 1.5646

S = 0.1254 R-Sq = 7.56% R-Sq(adj) = 4.54%
59

Figure 28. Intelligibility scores (RA) for Scenario 14.

6.2 Additional Comparisons of Results

The structure of this experiment enabled some additional comparisons. Those include evaluating 

possible differences between the two different PASS alarms, and between the mask with a vox 

port and the mask with an internal microphone. Further, changes in best practices and changes in 

the vocoder between the test in [6] and the current test can be evaluated.

6.2.1 Comparison of Two PASS Alarms

Two comparisons can be made from the current test data regarding the use of different PASS 

alarms. The first is between Scenarios 4 and 5, and the second is between Scenarios 6 and 7. 

Table 19 contains the intelligibility scores for this comparison. The detailed Minitab reports are 

in Appendix A. Figure 29 contains the bar chart of results for Scenarios 4 and 5, and Figure 30

contains the bar chart of results for Scenarios 6 and 7. The ANOVA test did not show any 

significant differences in intelligibility between the scenarios using PASS 1 and those using 

PASS 2.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

Intelligibility Scores (RA) for Scenario 14
60

Table 19. Intelligibility scores for PASS alarm comparison

Scenario 25 kHz AFM 12.5 kHz AFM P25 Full Rate P25 Half Rate

PASS 1 – Vox Port 0.606 0.586 0.499 0.508

PASS 2 – Vox Port 0.554 0.550 0.485 0.481

PASS 1 – Int Mic 0.800 0.822 0.707 0.710

PASS 2 – Int Mic 0.817 0.817 0.715 0.711

Figure 29. Intelligibility scores for PASS 1 and PASS 2 for a mask with a vox port.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Comparing PASS Alarms on Mask with Vox Port

PASS1

PASS2
61

Figure 30. Intelligibility scores for PASS 1 and PASS 2 for a mask with internal mic.

6.2.2 Comparison of SCBA Masks

Five scenarios were repeated with two SCBA masks. One of the masks used a microphone held 

to the vox port on the mask, and the other used an internal microphone. Comparisons can be 

made between Scenarios 2 and 3, 4 and 6, 5 and 7, 10 and 11, and 12 and 13. 

Table 20 contains the intelligibility scores for this comparison. The detailed Minitab reports are 

in Appendix A. Figures Figure 32 through Figure 35 contain the bar chart of results for each 

comparison. In all cases, the use of an internal microphone instead of the vox port resulted in a 

significant improvement in intelligibility for the analog systems. For the digital systems, a 

significant improvement was observed for those conditions with background noise. Differences 

in intelligibility for digital systems were not determined to be significant for those cases without 

background noise.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Comparing PASS Alarms on Mask with Internal Mic

PASS1

PASS2
62

Table 20. Intelligibility scores for comparing vox port to internal microphone

Scenario 25 kHz AFM 12.5 kHz 

AFM

P25 Full 

Rate

P25 Half 

Rate

Vox Port – No Noise 0.754 0.771 0.692 0.672

Int Mic – No noise 0.840 0.831 0.712 0.723

Vox Port - PASS 1 0.606 0.586 0.499 0.508

Int Mic – PASS 1 0.800 0.822 0.707 0.710

Vox Port – PASS 2 0.554 0.550 0.485 0.481

Int Mic – PASS 2 0.817 0.817 0.715 0.711

Vox Port – No Noise – Degraded Channel 0.484 0.409 0.694 0.679

Int Mic – No Noise – Degraded Channel 0.614 0.542 0.728 0.730

Vox Port – PASS 1 – Degraded Channel 0.339 0.328 0.488 0.498

Int Mic – PASS 1 – Degraded Channel --- --- 0.718 0.711

Figure 31. Intelligibility scores for SCBA mask with a vox port or internal microphone.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Comparing SCBAs with no Background Noise

Vox Port

Int Mic
63

Figure 32. SCBA mask comparison with PASS 1 background noise.

Figure 33. SCBA mask comparison with PASS 2 background noise.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Comparing SCBAs with PASS1 Background Noise

Vox Port

Int Mic

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Comparing SCBAs with PASS2 Background Noise

Vox Port

Int Mic
64

Figure 34. SCBA mask comparison with degraded channel.

Figure 35. SCBA mask comparison with degraded channel and PASS 1 background noise.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Vox Port

Int Mic

Comparing SCBAs with Degraded Channel, no Background Noise

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM P25 FR P25 HR

Intelligibility Score

System

Vox Port

Int Mic

Comparing SCBAs with Degraded Channel, PASS1 Background Noise
65

6.2.3 Differences Between Previous and Current Test

Initially, the IAFC recommended that field users hold the radio microphone one to two inches 

away from the voice port on their mask. Based on the results of our previous test, reported in [6], 

the IAFC updated their recommendations to hold the microphone directly against the voice 

port.

12 In the previous test, the microphone was placed one inch away from the SCBA vox port. 

In the current test, the microphone was placed directly against the SCBA vox port. When the 

previous test was performed, the most current version of the P25 voice codec was used and can 

be referred to as “Enhanced Full Rate” (EFR) or QFB (“Q” for QPSK-c modulation, including

C4FM and CQPSK, “F” for full rate, and “B” for enhanced). QFB was an improvement over the 

previously available Baseline Full Rate voice coder. The voice codec used in the current test, 

software version 1.40e, includes improvements to QFB. Table 21 shows additional differences 

between the previous test and the current test.

Table 21. Experiment configuration details for the previous and current MRT.

Previous Test Value Current Test Value

Distance from vox port to microphone 1 inch 0 inches

Delivered noise-free speech level 82 dBA 83.8-84 dBA

Ambient pink noise level 69.4-69.7 dBA 64.9-65 dBA

Radio implementation method Hardware Software

P25 voice codec implementation QFB Software version 1.40e

With these differences in mind, it is possible to compare three scenarios that were similar in both 

tests. In each test there was a scenario where no SCBA was used and no fireground noise was 

added, a scenario where an SCBA and its vox port were used and no fireground noise was added 

and a scenario where an SCBA and its vox port were used in the presence of a PASS alarm. Both 

studies tested 25 kHz analog, 12.5 kHz analog and P25 Full Rate in the first two scenarios. Only 

the current study tested 12.5 kHz analog in the third scenario.

The intelligibility scores for the analog radio systems in both tests are shown in Table 22. Figure 

36 displays the intelligibility scores graphically. Table 23 and Figure 37 show the intelligibility 

scores achieved by P25 radio systems in both tests.

 

12 Official documents [7] and [8] actually specify microphone placement to be “1 to 2 inches from the mouth or 

SCBA voice port,” but an instructional video (linked directly from the IAFC website and available here: 

http://business.motorola.com/publicsafety/SayItLoud/SayItLoud.html) instructs practitioners to place a microphone 

directly against the voice port of the SCBA.
66

Table 22. Intelligibility scores achieved in three different scenarios by AFM radios in both the 

previous and current test.

Scenario Previous Test Current Test

No Mask, No Noise 25 kHz Analog 0.881 0.896

12.5 kHz Analog 0.886 0.907

Mask (Vox Port), No Noise 25 kHz Analog 0.785 0.754

12.5 kHz Analog 0.798 0.771

Mask (Vox Port), PASS 1 25 kHz Analog 0.581 0.606

12.5 kHz Analog --- 0.586

Figure 36. Graphical representation of intelligibility scores achieved in three different scenarios 

by AFM systems in both the previous and current test.

Table 23. Intelligibility scores achieved in three different scenarios by P25 systems in both the 

previous and current test.

Scenario Previous Test Current Test

No Mask, No Noise P25 Full Rate 0.800 0.866

Mask (Vox Port), No Noise P25 Full Rate 0.591 0.692

Mask (Vox Port), PASS 1 P25 Full Rate 0.206 0.499

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

25 kHz AFM 12.5 kHz AFM 25 kHz AFM 12.5 kHz AFM 25 kHz AFM 12.5 kHz AFM

No Mask, No Noise Mask (Vox Port), No Noise Mask (Vox Port), PASS 1

Intelligibility Score

System

Previous Test

Current Test
67

Figure 37. Graphical representation of intelligibility scores achieved in three different scenarios 

by P25 systems in both the previous and current test.

For the no mask, no noise scenario, in the previous experiment there was a statistically 

significant difference between the P25 Full Rate systems and the two AFM systems, with the 

P25 Full Rate system having lower intelligibility. In the current test, the 25 kHz AFM and the 

P25 systems were statistically similar.

For the mask, no noise scenario, in the previous experiment there was a statistically significant 

difference between the P25 Full Rate systems and the two analog systems, with the P25 Full 

Rate system having lower intelligibility. In the current test, the three systems were not 

statistically significantly different.

For the mask and PASS alarm scenario, in the previous experiment there was a statistically 

significant difference between the P25 Full Rate systems and 25 kHz AFM, with the P25 Full 

Rate system having lower intelligibility. In the current test, both P25 systems were statistically 

similar to the 12.5 kHz AFM, but statistically different from the 25 kHz AFM, having a lower 

score. The 25 kHz AFM and 12.5 kHz AFM systems were statistically similar.

The results show that for analog systems, the cumulative effect of all the changes between the 

two tests resulted in changes in intelligibility score ranging from a decrease by 0.031 RA units to 

an increase by 0.025 RA units The similarity of these intelligibility scores may validate the 

software implementation method. For P25 Full Rate, the cumulative effect of all changes 

resulted in an intelligibility score increase by 0.066 RA units for the no mask, no noise scenario, 

an increase by 0.101 RA units for the mask, no noise scenario and an increase by 0.293 RA units 

for the mask and PASS alarm noise scenario. The magnitude of the change of intelligibility 

scores for P25 systems ranges from 2 to 10 times the size of the magnitude of the change of 

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

P25 Full Rate P25 Full Rate P25 Full Rate

No Mask, No Noise Mask (Vox Port), No Noise Mask (Vox Port), PASS 1

Intelligibility Score

System

Previous Test

Current Test
68

intelligibility scores for AFM systems. There are four changes between the two tests that are 

common to both the 25 kHz AFM and the P25 systems: the decrease in distance from the vox 

port to the microphone, the increase in delivered speech volume, the decrease in pink noise 

volume, and the switch from hardware to software implementations. Because of these four 

commonalities, it may be possible to attribute the P25 performance increases to the combination 

of the improved voice codec and the shorter distance between the vox port and microphone. 

6.3 Summary of Observations

For the clean channel condition in scenarios 1, 2, 4, and 5, the statistical comparison showed a 

near equivalence between the two AFM systems and the two P25 systems. In the case of 

Scenario 5, the equivalence was for all four systems, while in the other three scenarios there was 

significant overlap to equate one or more of the systems with all the others. Three of these four 

scenarios used the mask with the vox port. The other four clean channel conditions in Scenarios 

3, 6, 7, and 8 showed a clear separation, with the two analog systems scoring higher than the two 

P25 systems. It can be noted that three of these four scenarios used the mask with the internal 

microphone.

The largest difference in this test was observed in the night club noise scenario. This is likely due 

to the similarity in spectra between the speech and the background noise (music and talker 

babble).

For the degraded channel Scenarios 9, 10, 11 and 12, the two P25 systems were statistically 

better than the two analog systems. In Scenario 9, the 12.5 kHz analog system was significantly 

worse than the other three systems. The last degraded channel scenario, Scenario 14 with

nightclub noise, showed statistical equivalence for all four systems. 

For all Scenarios, the P25 full rate and P25 half-rate systems were considered equivalent, 

although there were two cases (Scenarios 1 and 2) where the small difference between the 

systems caused one of the P25 systems to be similar to one of the analog systems.

Additionally, the use of different PASS alarms had no observed significant impact on the 

intelligibility of the audio, but the use of a mask with an internal microphone improved 

intelligibility in all tested cases for the analog systems, and in most cases for the P25 systems.

Compared with the results obtained in [6], the performance of P25 Full Rate systems seems to 

have improved. This may be due to the cumulative effect of all the differences between the 

previous and current test, including experimental setup, an improved voice codec, and updated 

best practices. 
69

7 CONCLUSION

The effort practitioners and manufacturers are putting into improving intelligibility is making a 

difference. This is fully realized by looking at all aspects of the system and making a series of 

small improvements that ultimately result in significant improvements. This experiment has 

documented the results of two such improvements, as well as performing an initial intelligibility 

baseline study of the P25 half-rate vocoder. 

Some areas for potential future study might include noise cancelling microphones and other 

noise mitigating accessories. Also, it can be useful to obtain a better understanding of the impact 

that a range of channel impairment levels has on intelligibility.

Overall, since 2008, progress has been made to improve intelligibility in high-background-noise 

environments. With continued effort, there are promises of even more improvements. This will 

result in better communication for public safety practitioners, improving their safety and making 

them more effective.
70

8 ACKNOWLEDGEMENTS

The authors of this report wish to express gratitude to the following communities/agencies for 

adjusting schedules and allowing their public safety personnel to travel to Boulder to participate 

in the experiment:

 Boulder County Colorado Sheriff’s Office

 City of Boise Idaho Fire Department

 Eagle Idaho Fire Department

 Florida Department of Law Enforcement

 Littleton Colorado Fire Department

 Morris Illinois Fire Department

 New Jersey Police Task Force

 Oakland California Fire Department

 Ohio Multi-Agency Radio Communications (MARCS)

 Orlando Florida Fire Department

 Orleans County New York

 Pickaway County Ohio Sheriff’s Office

 Plainfield Indiana Fire Department

 State of Colorado

 Texas Department of Transportation

 Utah Communications Agency Network

The authors also thank the representatives of Motorola Solutions and Harris RF, manufacturers 

of public safety communications equipment, who provided equipment, expertise, and software

during testing and participated in the review of this report.
71

9 REFERENCES

[1] TIA-102.BABA (December 2003), “Project 25 - Vocoder Description,” 

Telecommunications Industry Association, Arlington, VA.

[2] TIA-102.BABA-1 (July 2009), “Project 25 Half-Rate Vocoder Annex,” 

Telecommunications Industry Association, Arlington, VA.

[3] TSB-102.BABE (May 2007), “Project 25 Vocoder Evaluation Mean Opinion Score Test,” 

Telecommunications Industry Association, Arlington, VA.

[4] TSB-102.BABF (March 2008), “Experiment 3 MOS Test Plan for Vocoder Technology for 

Project 25 Phase 2,” Telecommunications Industry Association, Arlington, VA.

[5] TIA-102.BABG (March 2010), “Project 25 Enhanced Vocoder Methods of Measurement 

for Performance,” Telecommunications Industry Association, Arlington, VA.

[6] David J. Atkinson and Andrew Catellier, “Intelligibility of selected radio systems in the 

presence of fireground noise: Test plan and results,” NTIA Technical Report TR-08-453, 

June 2008. Available http://www.its.bldrdoc.gov/publications/2490.aspx. 

[7] Interim Report and Recommendations: Fireground Noise and Digital Radio Transmissions, 

International Association of Fire Chiefs, Digital Project Working Group, May 2008. 

Available http://www.iafc.org/files/digProj_DPWGinterimReport.pdf. 

[8] Portable Radio Best Practices, International Association of Fire Chiefs, Digital Project 

Working Group, May 2008. Available 

http://www.iafc.org/files/digProb_PortableRadioBestPractices.pdf. 

[9] ANSI/ASA S3.2-2009, “Method for Measuring the Intelligibility of Speech Over 

Communication Systems,” American National Standards Institute, Washington, DC.

[10] NFPA 1981: Standard on Open-Circuit Self-Contained Breathing Apparatus (SCBA) for 

Emergency Services, 2007 Edition, National Fire Protection Association, Quincy, MA. 

[11] TIA-102.BABB (December 2003), “Project 25 - Mean Opinion Score Conformance Test,” 

Telecommunications Industry Association, Arlington, VA.

[12] TIA/EIA-603-A (August 2001), “Land Mobile FM or PM, Communications Equipment, 

Measurement and Performance Standards,” Telecommunications Industry Association, 

Arlington, VA.

[13] NFPA 1982: Standard on Personal Alert Safety Systems (PASS), 2007 Edition, National 

Fire Protection Association, Quincy, MA. 

[14] William H. Beyer, Standard Probability and Statistics Tables and Formulae, Boca Raton, 

FL: CRC Press, 1991.
72

[15] ITU-T Recommendation P.800 (08/1996), “Methods for subjective determination of 

transmission quality,” International Telecommunication Union, Geneva, Switzerland.

[16] ITU-T Recommendation G.191 (03/2010), “Software tools for speech and audio coding 

standardization,” International Telecommunication Union, Geneva, Switzerland.

[17] ITU-T Recommendation P.56 (12/2011), “Objective measurement of active speech level,” 

International Telecommunication Union, Geneva, Switzerland.

[18] ITU-T Recommendation P.57 (12/2011), “Artificial ears,” International 

Telecommunication Union, Geneva, Switzerland.

[19] ITU-T Recommendation P.58 (12/2011), “Head and torso simulator for telephonometry,”

International Telecommunication Union, Geneva, Switzerland.

[20] ANSI S1.4-1983 (R2006)/ANSI S1.4a-1985 (R2006), “American National Standard 

Specification for Sound Level Meters,” American National Standards Institute, 

Washington, DC.

[21] ANSI S1.4-1983 (R2006)/ANSI S1.4a-1985 (R2006), “Design Response of Weighting 

Networks for Acoustical Measurements,” American National Standards Institute, 

Washington, DC.
73

APPENDIX A: CONDITION LABELS AND MINITAB RESULTS

This Appendix contains the condition labels and Minitab results for this experiment. Table A-1

contains the list of condition labels for this experiment. These labels are used in the file naming 

and are correlated to the scenario numbers used in reporting results in Section 6.

Table A-1 List of condition labels used for this experiment.

Condition 

Label

Reference 

System

RF 

Channel Background Noise

c01 25 kHz AFM Clean No mask, no background noise

c02 25 kHz AFM Clean Mask (vox port), no background noise

c03 25 kHz AFM Clean Mask (int mic), no background noise

c04 25 kHz AFM Clean Mask (vox port), PASS 1 (-2 dB SNR)

c05 25 kHz AFM Clean Mask (vox port), PASS 2 (-2 dB SNR)

c06 25 kHz AFM Clean Mask (int mic), PASS 1 (-2 dB SNR)

c07 25 kHz AFM Clean Mask (int mic), PASS 2 (-2 dB SNR)

c08 25 kHz AFM Clean No mask, night club noise (5 db SNR)

c09 25 kHz AFM Static Degraded No mask, no background noise

c10 25 kHz AFM Static Degraded Mask (vox port), no background noise

c11 25 kHz AFM Static Degraded Mask (int mic), no background noise

c12 25 kHz AFM Static Degraded Mask (vox port), PASS 1 (-2 dB SNR)

c13 25 kHz AFM Static Degraded Mask (int mic), PASS 1 (-2 dB SNR) (creation error)

c14 25 kHz AFM Static Degraded No mask, night club noise (5 db SNR)

c15 12.5 kHz AFM Clean No mask, no background noise

c16 12.5 kHz AFM Clean Mask (vox port), no background noise

c17 12.5 kHz AFM Clean Mask (int mic), no background noise

c18 12.5 kHz AFM Clean Mask (vox port), PASS 1 (-2 dB SNR)

c19 12.5 kHz AFM Clean Mask (vox port), PASS 2 (-2 dB SNR)

c20 12.5 kHz AFM Clean Mask (int mic), PASS 1 (-2 dB SNR)

c21 12.5 kHz AFM Clean Mask (int mic), PASS 2 (-2 dB SNR)

c22 12.5 kHz AFM Clean No mask, night club noise (5 db SNR)

c23 12.5 kHz AFM Static Degraded No mask, no background noise

c24 12.5 kHz AFM Static Degraded Mask (vox port), no background noise

c25 12.5 kHz AFM Static Degraded Mask (int mic), no background noise

c26 12.5 kHz AFM Static Degraded Mask (vox port), PASS 1 (-2 dB SNR)

c27 12.5 kHz AFM Static Degraded Mask (int mic), PASS 1 (-2 dB SNR) (creation error)

c28 12.5 kHz AFM Static Degraded No mask, night club noise (5 db SNR)

c29 P25 FR Clean No mask, no background noise
74

Condition 

Label

Reference 

System

RF 

Channel Background Noise

c30 P25 FR Clean Mask (vox port), no background noise

c31 P25 FR Clean Mask (int mic), no background noise

c32 P25 FR Clean Mask (vox port), PASS 1 (-2 dB SNR)

c33 P25 FR Clean Mask (vox port), PASS 2 (-2 dB SNR)

c34 P25 FR Clean Mask (int mic), PASS 1 (-2 dB SNR)

c35 P25 FR Clean Mask (int mic), PASS 2 (-2 dB SNR)

c36 P25 FR Clean No mask, night club noise (5 db SNR)

c37 P25 FR Static Degraded No mask, no background noise

c38 P25 FR Static Degraded Mask (vox port), no background noise

c39 P25 FR Static Degraded Mask (int mic), no background noise

c40 P25 FR Static Degraded Mask (vox port), PASS 1 (-2 dB SNR)

c41 P25 FR Static Degraded Mask (int mic), PASS 1 (-2 dB SNR)

c42 P25 FR Static Degraded No mask, night club noise (5 db SNR)

c43 P25 HR Clean No mask, no background noise

c44 P25 HR Clean Mask (vox port), no background noise

c45 P25 HR Clean Mask (int mic), no background noise

c46 P25 HR Clean Mask (vox port), PASS 1 (-2 dB SNR)

c47 P25 HR Clean Mask (vox port), PASS 2 (-2 dB SNR)

c48 P25 HR Clean Mask (int mic), PASS 1 (-2 dB SNR)

c49 P25 HR Clean Mask (int mic), PASS 2 (-2 dB SNR)

c50 P25 HR Clean No mask, night club noise (5 db SNR)

c51 P25 HR Static Degraded No mask, no background noise

c52 P25 HR Static Degraded Mask (vox port), no background noise

c53 P25 HR Static Degraded Mask (int mic), no background noise

c54 P25 HR Static Degraded Mask (vox port), PASS 1 (-2 dB SNR)

c55 P25 HR Static Degraded Mask (int mic), PASS 1 (-2 dB SNR)

c56 P25 HR Static Degraded No mask, night club noise (5 db SNR)
75

A.1 Minitab Report for Scenario 1

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.06082 0.02027 5.83 0.001

Error 92 0.32011 0.00348

Total 95 0.38093

S = 0.05899 R-Sq = 15.97% R-Sq(adj) = 13.22%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --------+---------+---------+---------+-

1 24 0.89600 0.04835 (--------*--------)

2 24 0.90700 0.04970 (--------*--------)

3 24 0.86600 0.06483 (--------*--------)

4 24 0.84300 0.07005 (--------*--------)

 --------+---------+---------+---------+-

 0.840 0.875 0.910 0.945

Pooled StDev = 0.05899

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper

2 -0.04354 0.01100 0.06554

3 -0.08454 -0.03000 0.02454

4 -0.10754 -0.05300 0.00154

System +---------+---------+---------+---------

2 (--------*--------)

3 (--------*--------)

4 (--------*--------)

 +---------+---------+---------+---------

 -0.120 -0.060 0.000 0.060

System = 2 subtracted from:

System Lower Center Upper

3 -0.09554 -0.04100 0.01354

4 -0.11854 -0.06400 -0.00946

System +---------+---------+---------+---------

3 (--------*--------)

4 (--------*--------)

 +---------+---------+---------+---------

 -0.120 -0.060 0.000 0.060

System = 3 subtracted from:

System Lower Center Upper

4 -0.07754 -0.02300 0.03154

System +---------+---------+---------+---------

4 (--------*--------)

 +---------+---------+---------+---------

 -0.120 -0.060 0.000 0.060
76

A.2 Minitab Report for Scenario 2

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.16379 0.05460 6.55 0.000

Error 92 0.76673 0.00833

Total 95 0.93052

S = 0.09129 R-Sq = 17.60% R-Sq(adj) = 14.92%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

1 24 0.75400 0.09940 (---------*---------)

2 24 0.77100 0.06897 (---------*---------)

3 24 0.69200 0.10050 (--------*---------)

4 24 0.67200 0.09273 (--------*---------)

 -----+---------+---------+---------+----

 0.650 0.700 0.750 0.800

Pooled StDev = 0.09129

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

2 -0.06741 0.01700 0.10141 (--------*-------)

3 -0.14641 -0.06200 0.02241 (--------*-------)

4 -0.16641 -0.08200 0.00241 (--------*-------)

 --------+---------+---------+---------+-

 -0.10 0.00 0.10 0.20

System = 2 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

3 -0.16341 -0.07900 0.00541 (-------*--------)

4 -0.18341 -0.09900 -0.01459 (-------*--------)

 --------+---------+---------+---------+-

 -0.10 0.00 0.10 0.20

System = 3 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

4 -0.10441 -0.02000 0.06441 (-------*-------)

 --------+---------+---------+---------+-

 -0.10 0.00 0.10 0.20
77

A.3 Minitab Report for Scenario 3

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.33660 0.11220 19.18 0.000

Error 92 0.53813 0.00585

Total 95 0.87473

S = 0.07648 R-Sq = 38.48% R-Sq(adj) = 36.47%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --------+---------+---------+---------+-

1 24 0.84000 0.08542 (------*------)

2 24 0.83100 0.05549 (------*-----)

3 24 0.71200 0.08754 (------*------)

4 24 0.72300 0.07320 (------*-----)

 --------+---------+---------+---------+-

 0.720 0.780 0.840 0.900

Pooled StDev = 0.07648

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper

2 -0.07972 -0.00900 0.06172

3 -0.19872 -0.12800 -0.05728

4 -0.18772 -0.11700 -0.04628

System +---------+---------+---------+---------

2 (------*------)

3 (------*------)

4 (------*------)

 +---------+---------+---------+---------

 -0.20 -0.10 0.00 0.10

System = 2 subtracted from:

System Lower Center Upper

3 -0.18972 -0.11900 -0.04828

4 -0.17872 -0.10800 -0.03728

System +---------+---------+---------+---------

3 (------*------)

4 (------*------)

 +---------+---------+---------+---------

 -0.20 -0.10 0.00 0.10

System = 3 subtracted from:

System Lower Center Upper +---------+---------+---------+---------

4 -0.05972 0.01100 0.08172 (------*------)

 +---------+---------+---------+---------

 -0.20 -0.10 0.00 0.10
78

A.4 Minitab Report for Scenario 4

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.21112 0.07037 7.06 0.000

Error 92 0.91754 0.00997

Total 95 1.12867

S = 0.09987 R-Sq = 18.71% R-Sq(adj) = 16.05%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ------+---------+---------+---------+---

1 24 0.60600 0.09413 (--------*--------)

2 24 0.58600 0.11218 (--------*--------)

3 24 0.49900 0.08353 (--------*--------)

4 24 0.50800 0.10710 (--------*--------)

 ------+---------+---------+---------+---

 0.480 0.540 0.600 0.660

Pooled StDev = 0.09987

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper

2 -0.11234 -0.02000 0.07234

3 -0.19934 -0.10700 -0.01466

4 -0.19034 -0.09800 -0.00566

System +---------+---------+---------+---------

2 (--------*--------)

3 (--------*---------)

4 (--------*--------)

 +---------+---------+---------+---------

 -0.20 -0.10 0.00 0.10

System = 2 subtracted from:

0.5

3 -0.17934 -0.08700 0.00534

4 -0.17034 -0.07800 0.01434

System +---------+---------+---------+---------

3 (--------*---------)

4 (--------*--------)

 +---------+---------+---------+---------

 -0.20 -0.10 0.00 0.10

System = 3 subtracted from:

System Lower Center Upper +---------+---------+---------+---------

4 -0.08334 0.00900 0.10134 (--------*--------)

 +---------+---------+---------+---------

 -0.20 -0.10 0.00 0.10
79

A.5 Minitab Report for Scenario 5

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.1146 0.0382 3.30 0.024

Error 92 1.0644 0.0116

Total 95 1.1790

S = 0.1076 R-Sq = 9.72% R-Sq(adj) = 6.78%

 Individual 99% CIs For Mean Based on

Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

1 24 0.5540 0.1061 (-----------*----------)

2 24 0.5500 0.1060 (-----------*-----------)

3 24 0.4850 0.1011 (-----------*-----------)

4 24 0.4810 0.1165 (----------*-----------)

 -----+---------+---------+---------+----

 0.450 0.500 0.550 0.600

Pooled StDev = 0.1076

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

2 -0.1035 -0.0040 0.0955 (---------*---------)

3 -0.1685 -0.0690 0.0305 (---------*---------)

4 -0.1725 -0.0730 0.0265 (---------*---------)

 -------+---------+---------+---------+--

 -0.10 0.00 0.10 0.20

System = 2 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

3 -0.1645 -0.0650 0.0345 (--------*---------)

4 -0.1685 -0.0690 0.0305 (---------*---------)

 -------+---------+---------+---------+--

 -0.10 0.00 0.10 0.20

System = 3 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

4 -0.1035 -0.0040 0.0955 (---------*---------)

 -------+---------+---------+---------+--

 -0.10 0.00 0.10 0.20
80

A.6 Minitab Report for Scenario 6

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.25807 0.08602 16.78 0.000

Error 92 0.47172 0.00513

Total 95 0.72979

S = 0.07161 R-Sq = 35.36% R-Sq(adj) = 33.25%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ------+---------+---------+---------+---

1 24 0.80000 0.07411 (-------*-------)

2 24 0.82200 0.06405 (------*-------)

3 24 0.70700 0.07880 (------*-------)

4 24 0.71000 0.06859 (-------*-------)

 ------+---------+---------+---------+---

 0.700 0.750 0.800 0.850

Pooled StDev = 0.07161

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

2 -0.04421 0.02200 0.08821 (-----*------)

3 -0.15921 -0.09300 -0.02679 (------*-----)

4 -0.15621 -0.09000 -0.02379 (------*------)

 --------+---------+---------+---------+-

 -0.10 0.00 0.10 0.20

System = 2 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

3 -0.18121 -0.11500 -0.04879 (-----*------)

4 -0.17821 -0.11200 -0.04579 (------*-----)

 --------+---------+---------+---------+-

 -0.10 0.00 0.10 0.20

System = 3 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

4 -0.06321 0.00300 0.06921 (-----*------)

 --------+---------+---------+---------+-

 -0.10 0.00 0.10 0.20
81

A.7 Minitab Report for Scenario 7

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.25978 0.08659 17.23 0.000

Error 92 0.46243 0.00503

Total 95 0.72221

S = 0.07090 R-Sq = 35.97% R-Sq(adj) = 33.88%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

1 24 0.81700 0.08777 (------*-------)

2 24 0.81700 0.07747 (------*-------)

3 24 0.71500 0.05496 (-------*-------)

4 24 0.71100 0.05814 (------*-------)

 -----+---------+---------+---------+----

 0.700 0.750 0.800 0.850

Pooled StDev = 0.07090

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

2 -0.06556 -0.00000 0.06556 (------*------)

3 -0.16756 -0.10200 -0.03644 (------*-----)

4 -0.17156 -0.10600 -0.04044 (-----*------)

 -------+---------+---------+---------+--

 -0.10 0.00 0.10 0.20

System = 2 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

3 -0.16756 -0.10200 -0.03644 (------*-----)

4 -0.17156 -0.10600 -0.04044 (-----*------)

 -------+---------+---------+---------+--

 -0.10 0.00 0.10 0.20

System = 3 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

4 -0.06956 -0.00400 0.06156 (------*-----)

 -------+---------+---------+---------+--

 -0.10 0.00 0.10 0.20
82

A.8 Minitab Report for Scenario 8

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.7115 0.2372 16.20 0.000

Error 92 1.3467 0.0146

Total 95 2.0582

S = 0.1210 R-Sq = 34.57% R-Sq(adj) = 32.43%

 Individual 99% CIs For Mean Based on

Pooled StDev

Level N Mean StDev ------+---------+---------+---------+---

1 24 0.6830 0.1148 (-------*-------)

2 24 0.6560 0.0992 (-------*-------)

3 24 0.4940 0.1454 (-------*-------)

4 24 0.5030 0.1200 (-------*-------)

 ------+---------+---------+---------+---

 0.480 0.560 0.640 0.720

Pooled StDev = 0.1210

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper +---------+---------+---------+---------

2 -0.1389 -0.0270 0.0849 (------*-------)

3 -0.3009 -0.1890 -0.0771 (------*-------)

4 -0.2919 -0.1800 -0.0681 (------*------)

 +---------+---------+---------+---------

 -0.30 -0.15 0.00 0.15

System = 2 subtracted from:

System Lower Center Upper +---------+---------+---------+---------

3 -0.2739 -0.1620 -0.0501 (------*-------)

4 -0.2649 -0.1530 -0.0411 (-------*------)

 +---------+---------+---------+---------

 -0.30 -0.15 0.00 0.15

System = 3 subtracted from:

System Lower Center Upper +---------+---------+---------+---------

4 -0.1029 0.0090 0.1209 (-------*------)

 +---------+---------+---------+---------

 -0.30 -0.15 0.00 0.15
83

A.9 Minitab Report for Scenario 9

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 1.30058 0.43353 57.49 0.000

Error 92 0.69377 0.00754

Total 95 1.99435

S = 0.08684 R-Sq = 65.21% R-Sq(adj) = 64.08%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ------+---------+---------+---------+---

1 24 0.70100 0.08979 (----*----)

2 24 0.59000 0.12009 (----*----)

3 24 0.87600 0.05233 (----*---)

4 24 0.85200 0.07030 (---*----)

 ------+---------+---------+---------+---

 0.60 0.70 0.80 0.90

Pooled StDev = 0.08684

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

2 -0.19130 -0.11100 -0.03070 (---*---)

3 0.09470 0.17500 0.25530 (---*---)

4 0.07070 0.15100 0.23130 (---*---)

 --------+---------+---------+---------+-

 -0.20 0.00 0.20 0.40

System = 2 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

3 0.20570 0.28600 0.36630 (---*---)

4 0.18170 0.26200 0.34230 (---*---)

 --------+---------+---------+---------+-

 -0.20 0.00 0.20 0.40

System = 3 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

4 -0.10430 -0.02400 0.05630 (---*---)

 --------+---------+---------+---------+-

 -0.20 0.00 0.20 0.40
84

A.10 Minitab Report for Scenario 10

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 1.45260 0.48420 48.83 0.000

Error 92 0.91224 0.00992

Total 95 2.36484

S = 0.09958 R-Sq = 61.42% R-Sq(adj) = 60.17%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ----+---------+---------+---------+-----

1 24 0.48400 0.11948 (----*-----)

2 24 0.40900 0.11319 (----*----)

3 24 0.69400 0.07952 (----*-----)

4 24 0.67900 0.07907 (----*----)

 ----+---------+---------+---------+-----

 0.40 0.50 0.60 0.70

Pooled StDev = 0.09958

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper ---------+---------+---------+---------+

2 -0.16708 -0.07500 0.01708 (---*----)

3 0.11792 0.21000 0.30208 (----*---)

4 0.10292 0.19500 0.28708 (----*---)

 ---------+---------+---------+---------+

 -0.20 0.00 0.20 0.40

System = 2 subtracted from:

System Lower Center Upper ---------+---------+---------+---------+

3 0.19292 0.28500 0.37708 (---*----)

4 0.17792 0.27000 0.36208 (----*---)

 ---------+---------+---------+---------+

 -0.20 0.00 0.20 0.40

System = 3 subtracted from:

System Lower Center Upper ---------+---------+---------+---------+

4 -0.10708 -0.01500 0.07708 (---*----)

 ---------+---------+---------+---------+

 -0.20 0.00 0.20 0.40
85

A.11 Minitab Report for Scenario 11

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.60948 0.20316 24.52 0.000

Error 92 0.76234 0.00829

Total 95 1.37182

S = 0.09103 R-Sq = 44.43% R-Sq(adj) = 42.62%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --------+---------+---------+---------+-

1 24 0.61400 0.09440 (-----*-----)

2 24 0.54200 0.10778 (-----*-----)

3 24 0.72800 0.08716 (-----*-----)

4 24 0.73000 0.07086 (-----*-----)

 --------+---------+---------+---------+-

 0.560 0.640 0.720 0.800

Pooled StDev = 0.09103

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

2 -0.15617 -0.07200 0.01217 (----*-----)

3 0.02983 0.11400 0.19817 (-----*----)

4 0.03183 0.11600 0.20017 (-----*----)

 --------+---------+---------+---------+-

 -0.15 0.00 0.15 0.30

System = 2 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

3 0.10183 0.18600 0.27017 (----*-----)

4 0.10383 0.18800 0.27217 (-----*----)

 --------+---------+---------+---------+-

 -0.15 0.00 0.15 0.30

System = 3 subtracted from:

System Lower Center Upper --------+---------+---------+---------+-

4 -0.08217 0.00200 0.08617 (----*-----)

 --------+---------+---------+---------+-

 -0.15 0.00 0.15 0.30
86

A.12 Minitab Report for Scenario 12

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.61322 0.20441 20.52 0.000

Error 92 0.91649 0.00996

Total 95 1.52971

S = 0.09981 R-Sq = 40.09% R-Sq(adj) = 38.13%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -+---------+---------+---------+--------

1 24 0.33900 0.08979 (------*-------)

2 24 0.32800 0.09416 (-------*-------)

3 24 0.48800 0.11205 (-------*------)

4 24 0.49800 0.10180 (-------*-------)

 -+---------+---------+---------+--------

 0.280 0.350 0.420 0.490

Pooled StDev = 0.09981

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

2 -0.10329 -0.01100 0.08129 (-----*-----)

3 0.05671 0.14900 0.24129 (-----*-----)

4 0.06671 0.15900 0.25129 (------*-----)

 -------+---------+---------+---------+--

 -0.15 0.00 0.15 0.30

System = 2 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

3 0.06771 0.16000 0.25229 (-----*-----)

4 0.07771 0.17000 0.26229 (-----*-----)

 -------+---------+---------+---------+--

 -0.15 0.00 0.15 0.30

System = 3 subtracted from:

System Lower Center Upper -------+---------+---------+---------+--

4 -0.08229 0.01000 0.10229 (-----*-----)

 -------+---------+---------+---------+--

 -0.15 0.00 0.15 0.30
87

A.13 Minitab Report for Scenario 14

One-way ANOVA: RA versus System 

Source DF SS MS F P

System 3 0.1182 0.0394 2.51 0.064

Error 92 1.4464 0.0157

Total 95 1.5647

S = 0.1254 R-Sq = 7.56% R-Sq(adj) = 4.54%

 Individual 99% CIs For Mean Based on

Pooled StDev

Level N Mean StDev ----+---------+---------+---------+-----

1 24 0.5420 0.1173 (----------*-----------)

2 24 0.4700 0.1256 (----------*-----------)

3 24 0.5300 0.1331 (----------*-----------)

4 24 0.4630 0.1250 (----------*----------)

 ----+---------+---------+---------+-----

 0.420 0.480 0.540 0.600

Pooled StDev = 0.1254

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of System

Individual confidence level = 99.81%

System = 1 subtracted from:

System Lower Center Upper ---------+---------+---------+---------+

2 -0.1879 -0.0720 0.0439 (-----------*----------)

3 -0.1279 -0.0120 0.1039 (-----------*----------)

4 -0.1949 -0.0790 0.0369 (----------*-----------)

 ---------+---------+---------+---------+

 -0.10 0.00 0.10 0.20

System = 2 subtracted from:

System Lower Center Upper ---------+---------+---------+---------+

3 -0.0559 0.0600 0.1759 (-----------*-----------)

4 -0.1229 -0.0070 0.1089 (----------*-----------)

 ---------+---------+---------+---------+

 -0.10 0.00 0.10 0.20

System = 3 subtracted from:

System Lower Center Upper ---------+---------+---------+---------+

4 -0.1829 -0.0670 0.0489 (----------*-----------)

 ---------+---------+---------+---------+

 -0.10 0.00 0.10 0.20
88

A.14 Minitab Report for PASS Alarm Comparison Using Vox Port

Results for: 25 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.0324 0.0324 3.22 0.079

Error 46 0.4629 0.0101

Total 47 0.4954

S = 0.1003 R-Sq = 6.55% R-Sq(adj) = 4.52%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev +---------+---------+---------+---------

4 24 0.6060 0.0941 (----------*----------)

5 24 0.5540 0.1061 (----------*----------)

 +---------+---------+---------+---------

 0.500 0.550 0.600 0.650

Pooled StDev = 0.1003

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 4 subtracted from:

Scenario Lower Center Upper --+---------+---------+---------+-------

5 -0.1298 -0.0520 0.0258 (------------*------------)

 --+---------+---------+---------+-------

 -0.120 -0.060 0.000 0.060

Results for: 12.5 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.0156 0.0156 1.31 0.259

Error 46 0.5478 0.0119

Total 47 0.5633

S = 0.1091 R-Sq = 2.76% R-Sq(adj) = 0.65%

 Individual 99% CIs For Mean Based on

 Pooled StDev

Level N Mean StDev -------+---------+---------+---------+--

4 24 0.5860 0.1122 (-------------*--------------)

5 24 0.5500 0.1060 (-------------*--------------)

 -------+---------+---------+---------+--

 0.520 0.560 0.600 0.640

Pooled StDev = 0.1091

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
89

Scenario = 4 subtracted from:

Scenario Lower Center Upper +---------+---------+---------+---------

5 -0.1206 -0.0360 0.0486 (-------------*-------------)

 +---------+---------+---------+---------

 -0.120 -0.060 0.000 0.060

Results for: P25 Full Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.00235 0.00235 0.27 0.603

Error 46 0.39547 0.00860

Total 47 0.39782

S = 0.09272 R-Sq = 0.59% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

4 24 0.49900 0.08353 (----------------*----------------)

5 24 0.48500 0.10108 (----------------*----------------)

 -----+---------+---------+---------+----

 0.450 0.480 0.510 0.540

Pooled StDev = 0.09272

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 4 subtracted from:

Scenario Lower Center Upper

5 -0.08592 -0.01400 0.05792

Scenario -+---------+---------+---------+--------

5 (----------------*-----------------)

 -+---------+---------+---------+--------

 -0.080 -0.040 0.000 0.040

Results for: P25 Half-Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.0087 0.0087 0.70 0.407

Error 46 0.5758 0.0125

Total 47 0.5845

S = 0.1119 R-Sq = 1.50% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on

Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

4 24 0.5080 0.1071 (--------------*--------------)

5 24 0.4810 0.1165 (--------------*---------------)

 -----+---------+---------+---------+----

 0.440 0.480 0.520 0.560

Pooled StDev = 0.1119
90

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 4 subtracted from:

Scenario Lower Center Upper ---------+---------+---------+---------+

5 -0.1138 -0.0270 0.0598 (--------------*-------------)

 ---------+---------+---------+---------+

 -0.060 0.000 0.060 0.120
91

A.15 Minitab Report for PASS Alarm Comparison Using Internal Microphone

Results for: 25 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.00347 0.00347 0.53 0.472

Error 46 0.30353 0.00660

Total 47 0.30700

S = 0.08123 R-Sq = 1.13% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --------+---------+---------+---------+-

6 24 0.80000 0.07411 (--------------*--------------)

7 24 0.81700 0.08777 (--------------*--------------)

 --------+---------+---------+---------+-

 0.780 0.810 0.840 0.870

Pooled StDev = 0.08123

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 6 subtracted from:

Scenario Lower Center Upper ------+---------+---------+---------+---

7 -0.04601 0.01700 0.08001 (---------------*---------------)

 ------+---------+---------+---------+---

 -0.040 0.000 0.040 0.080

Results for: 12.5 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.00030 0.00030 0.06 0.809

Error 46 0.23239 0.00505

Total 47 0.23269

S = 0.07108 R-Sq = 0.13% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ---------+---------+---------+---------+

6 24 0.82200 0.06405 (---------------*--------------)

7 24 0.81700 0.07747 (---------------*--------------)

 ---------+---------+---------+---------+

 0.800 0.825 0.850 0.875

Pooled StDev = 0.07108

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
92

Scenario = 6 subtracted from:

Scenario Lower Center Upper

7 -0.06013 -0.00500 0.05013

Scenario +---------+---------+---------+---------

7 (-----------------*------------------)

 +---------+---------+---------+---------

 -0.060 -0.030 0.000 0.030

Results for: P25 Full Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.00077 0.00077 0.17 0.685

Error 46 0.21230 0.00462

Total 47 0.21307

S = 0.06794 R-Sq = 0.36% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --+---------+---------+---------+-------

6 24 0.70700 0.07880 (--------------*--------------)

7 24 0.71500 0.05496 (--------------*--------------)

 --+---------+---------+---------+-------

 0.675 0.700 0.725 0.750

Pooled StDev = 0.06794

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 6 subtracted from:

Scenario Lower Center Upper --------+---------+---------+---------+-

7 -0.04470 0.00800 0.06070 (-----------------*----------------)

 --------+---------+---------+---------+-

 -0.030 0.000 0.030 0.060

Results for: P25 Half Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.00001 0.00001 0.00 0.957

Error 46 0.18593 0.00404

Total 47 0.18594

S = 0.06358 R-Sq = 0.01% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --+---------+---------+---------+-------

6 24 0.71000 0.06859 (----------------*----------------)

7 24 0.71100 0.05814 (----------------*-----------------)

 --+---------+---------+---------+-------

 0.680 0.700 0.720 0.740

Pooled StDev = 0.06358

Tukey 99% Simultaneous Confidence Intervals
93

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 6 subtracted from:

Scenario Lower Center Upper

7 -0.04831 0.00100 0.05031

Scenario +---------+---------+---------+---------

7 (------------------*-------------------)

 +---------+---------+---------+---------

 -0.050 -0.025 0.000 0.025
94

A.16 Minitab Report for SCBA Comparison with no Background Noise

Results for: 25 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.08875 0.08875 10.33 0.002

Error 46 0.39504 0.00859

Total 47 0.48379

S = 0.09267 R-Sq = 18.35% R-Sq(adj) = 16.57%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ---------+---------+---------+---------+

2 24 0.75400 0.09940 (---------*---------)

3 24 0.84000 0.08542 (---------*---------)

 ---------+---------+---------+---------+

 0.750 0.800 0.850 0.900

Pooled StDev = 0.09267

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 2 subtracted from:

Scenario Lower Center Upper --+---------+---------+---------+-------

3 0.01412 0.08600 0.15788 (-----------*-----------)

 --+---------+---------+---------+-------

 -0.060 0.000 0.060 0.120

Results for: 12.5 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.04320 0.04320 11.03 0.002

Error 46 0.18024 0.00392

Total 47 0.22344

S = 0.06260 R-Sq = 19.33% R-Sq(adj) = 17.58%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev +---------+---------+---------+---------

2 24 0.77100 0.06897 (---------*---------)

3 24 0.83100 0.05549 (--------*---------)

 +---------+---------+---------+---------

 0.735 0.770 0.805 0.840

Pooled StDev = 0.06260

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
95

Scenario = 2 subtracted from:

Scenario Lower Center Upper --+---------+---------+---------+-------

3 0.01145 0.06000 0.10855 (-----------*-----------)

 --+---------+---------+---------+-------

 -0.040 0.000 0.040 0.080

Results for: P25 Full Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.00480 0.00480 0.54 0.466

Error 46 0.40858 0.00888

Total 47 0.41338

S = 0.09424 R-Sq = 1.16% R-Sq(adj) = 0.00%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -------+---------+---------+---------+--

2 24 0.69200 0.10050 (--------------*-------------)

3 24 0.71200 0.08754 (-------------*--------------)

 -------+---------+---------+---------+--

 0.665 0.700 0.735 0.770

Pooled StDev = 0.09424

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 2 subtracted from:

Scenario Lower Center Upper -----+---------+---------+---------+----

3 -0.05310 0.02000 0.09310 (--------------*--------------)

 -----+---------+---------+---------+----

 -0.050 0.000 0.050 0.100

Results for: P25 Half Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.03121 0.03121 4.47 0.040

Error 46 0.32100 0.00698

Total 47 0.35221

S = 0.08354 R-Sq = 8.86% R-Sq(adj) = 6.88%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ---+---------+---------+---------+------

2 24 0.67200 0.09273 (----------*----------)

3 24 0.72300 0.07320 (-----------*----------)

 ---+---------+---------+---------+------

 0.640 0.680 0.720 0.760

Pooled StDev = 0.08354

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
96

Scenario = 2 subtracted from:

Scenario Lower Center Upper ---+---------+---------+---------+------

3 -0.01380 0.05100 0.11580 (------------*------------)

 ---+---------+---------+---------+------

 -0.050 0.000 0.050 0.100
97

A.17 Minitab Report for SCBA Comparison with PASS 1 Background Noise

Results for: 25 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.45163 0.45163 62.93 0.000

Error 46 0.33014 0.00718

Total 47 0.78178

S = 0.08472 R-Sq = 57.77% R-Sq(adj) = 56.85%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev +---------+---------+---------+---------

4 24 0.60600 0.09413 (-----*-----)

6 24 0.80000 0.07411 (-----*-----)

 +---------+---------+---------+---------

 0.560 0.640 0.720 0.800

Pooled StDev = 0.08472

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 4 subtracted from:

Scenario Lower Center Upper -------+---------+---------+---------+--

6 0.12829 0.19400 0.25971 (-----*------)

 -------+---------+---------+---------+--

 0.00 0.10 0.20 0.30

Results for: 12.5 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.66835 0.66835 80.10 0.000

Error 46 0.38381 0.00834

Total 47 1.05216

S = 0.09134 R-Sq = 63.52% R-Sq(adj) = 62.73%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ------+---------+---------+---------+---

4 24 0.58600 0.11218 (----*----)

6 24 0.82200 0.06405 (----*----)

 ------+---------+---------+---------+---

 0.60 0.70 0.80 0.90

Pooled StDev = 0.09134

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
98

Scenario = 4 subtracted from:

Scenario Lower Center Upper -------+---------+---------+---------+--

6 0.16515 0.23600 0.30685 (------*------)

 -------+---------+---------+---------+--

 0.00 0.10 0.20 0.30

Results for: P25 Full Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.51917 0.51917 78.74 0.000

Error 46 0.30331 0.00659

Total 47 0.82248

S = 0.08120 R-Sq = 63.12% R-Sq(adj) = 62.32%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ---+---------+---------+---------+------

4 24 0.49900 0.08353 (----*-----)

6 24 0.70700 0.07880 (----*-----)

 ---+---------+---------+---------+------

 0.480 0.560 0.640 0.720

Pooled StDev = 0.08120

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 4 subtracted from:

Scenario Lower Center Upper ------+---------+---------+---------+---

6 0.14501 0.20800 0.27099 (-----*-----)

 ------+---------+---------+---------+---

 0.00 0.10 0.20 0.30

Results for: P25 Half Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.48965 0.48965 60.55 0.000

Error 46 0.37200 0.00809

Total 47 0.86165

S = 0.08993 R-Sq = 56.83% R-Sq(adj) = 55.89%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ---+---------+---------+---------+------

4 24 0.50800 0.10710 (-----*------)

6 24 0.71000 0.06859 (-----*-----)

 ---+---------+---------+---------+------

 0.480 0.560 0.640 0.720

Pooled StDev = 0.08993

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
99

Scenario = 4 subtracted from:

Scenario Lower Center Upper -------+---------+---------+---------+--

6 0.13225 0.20200 0.27175 (------*------)

 -------+---------+---------+---------+--

 0.00 0.10 0.20 0.30
100

A.18 Minitab Report for SCBA Comparison with PASS 2 Background Noise

Results for: 25 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.83003 0.83003 87.51 0.000

Error 46 0.43630 0.00948

Total 47 1.26632

S = 0.09739 R-Sq = 65.55% R-Sq(adj) = 64.80%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev +---------+---------+---------+---------

5 24 0.55400 0.10614 (----*-----)

7 24 0.81700 0.08777 (-----*----)

 +---------+---------+---------+---------

 0.50 0.60 0.70 0.80

Pooled StDev = 0.09739

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 5 subtracted from:

Scenario Lower Center Upper ------+---------+---------+---------+---

7 0.18746 0.26300 0.33854 (-----*-----)

 ------+---------+---------+---------+---

 0.00 0.12 0.24 0.36

Results for: 12.5 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.85547 0.85547 99.28 0.000

Error 46 0.39636 0.00862

Total 47 1.25183

S = 0.09283 R-Sq = 68.34% R-Sq(adj) = 67.65%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev +---------+---------+---------+---------

5 24 0.55000 0.10598 (----*----)

7 24 0.81700 0.07747 (----*----)

 +---------+---------+---------+---------

 0.50 0.60 0.70 0.80

Pooled StDev = 0.09283

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
101

Scenario = 5 subtracted from:

Scenario Lower Center Upper ------+---------+---------+---------+---

7 0.19500 0.26700 0.33900 (-----*-----)

 ------+---------+---------+---------+---

 0.00 0.12 0.24 0.36

Results for: P25 Full Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.63480 0.63480 95.91 0.000

Error 46 0.30446 0.00662

Total 47 0.93926

S = 0.08136 R-Sq = 67.58% R-Sq(adj) = 66.88%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

5 24 0.48500 0.10108 (-----*----)

7 24 0.71500 0.05496 (----*-----)

 -----+---------+---------+---------+----

 0.480 0.560 0.640 0.720

Pooled StDev = 0.08136

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 5 subtracted from:

Scenario Lower Center Upper ------+---------+---------+---------+---

7 0.16689 0.23000 0.29311 (-----*-----)

 ------+---------+---------+---------+---

 0.00 0.10 0.20 0.30

Results for: P25 Half Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.63480 0.63480 74.93 0.000

Error 46 0.38971 0.00847

Total 47 1.02451

S = 0.09204 R-Sq = 61.96% R-Sq(adj) = 61.13%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -------+---------+---------+---------+--

5 24 0.48100 0.11647 (----*----)

7 24 0.71100 0.05814 (----*----)

 -------+---------+---------+---------+--

 0.50 0.60 0.70 0.80

Pooled StDev = 0.09204

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
102

Scenario = 5 subtracted from:

Scenario Lower Center Upper -------+---------+---------+---------+--

7 0.15860 0.23000 0.30140 (------*------)

 -------+---------+---------+---------+--

 0.00 0.10 0.20 0.30
103

A.19 Minitab Report for SCBA Comparison with Degraded Channel and 

No Background Noise

Results for: 25 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.2028 0.2028 17.49 0.000

Error 46 0.5333 0.0116

Total 47 0.7361

S = 0.1077 R-Sq = 27.55% R-Sq(adj) = 25.98%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev ---------+---------+---------+---------+

10 24 0.4840 0.1195 (-------*--------)

11 24 0.6140 0.0944 (--------*-------)

 ---------+---------+---------+---------+

 0.490 0.560 0.630 0.700

Pooled StDev = 0.1077

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 10 subtracted from:

Scenario Lower Center Upper +---------+---------+---------+---------

11 0.0465 0.1300 0.2135 (---------*----------)

 +---------+---------+---------+---------

 -0.080 0.000 0.080 0.160

Results for: 12.5 kHz Analog

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.2123 0.2123 17.38 0.000

Error 46 0.5619 0.0122

Total 47 0.7741

S = 0.1105 R-Sq = 27.42% R-Sq(adj) = 25.84%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev +---------+---------+---------+---------

10 24 0.4090 0.1132 (-------*--------)

11 24 0.5420 0.1078 (-------*--------)

 +---------+---------+---------+---------

 0.350 0.420 0.490 0.560

Pooled StDev = 0.1105

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
104

Scenario = 10 subtracted from:

Scenario Lower Center Upper -+---------+---------+---------+--------

11 0.0473 0.1330 0.2187 (----------*---------)

 -+---------+---------+---------+--------

 -0.080 0.000 0.080 0.160

Results for: P25 Full Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.01387 0.01387 1.99 0.165

Error 46 0.32016 0.00696

Total 47 0.33403

S = 0.08343 R-Sq = 4.15% R-Sq(adj) = 2.07%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev -----+---------+---------+---------+----

10 24 0.69400 0.07952 (------------*------------)

11 24 0.72800 0.08716 (------------*------------)

 -----+---------+---------+---------+----

 0.665 0.700 0.735 0.770

Pooled StDev = 0.08343

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%

Scenario = 10 subtracted from:

Scenario Lower Center Upper ---+---------+---------+---------+------

11 -0.03071 0.03400 0.09871 (------------*------------)

 ---+---------+---------+---------+------

 -0.050 0.000 0.050 0.100

Results for: P25 Half Rate

One-way ANOVA: RA versus Scenario 

Source DF SS MS F P

Scenario 1 0.03121 0.03121 5.54 0.023

Error 46 0.25927 0.00564

Total 47 0.29048

S = 0.07508 R-Sq = 10.74% R-Sq(adj) = 8.80%

 Individual 99% CIs For Mean Based on Pooled StDev

Level N Mean StDev --------+---------+---------+---------+-

10 24 0.67900 0.07907 (-----------*-----------)

11 24 0.73000 0.07086 (-----------*----------)

 --------+---------+---------+---------+-

 0.665 0.700 0.735 0.770

Pooled StDev = 0.07508

Tukey 99% Simultaneous Confidence Intervals

All Pairwise Comparisons among Levels of Scenario

Individual confidence level = 99.00%
105

Scenario = 10 subtracted from:

Scenario Lower Center Upper --+---------+---------+---------+-------

11 -0.00723 0.05100 0.10923 (----------*-----------)

 --+---------+---------+---------+-------

 -0.050 0.000 0.050 0.100
106

APPENDIX B: LISTENER SCORES

Table B-1 tabulates the scores collected from the listener panels. Adjusted intelligibility scores 

(RA) are presented by talker, condition, and listener. Each RA in the table represents 50 trials in 

the listening test.

Table B-1. Listener scores by talker, condition, and listener.

Talker Condition Listener RA

F1 1 4 0.856

F1 1 21 0.952

F1 1 27 0.976

F1 1 30 0.880

F1 1 30 0.928

F1 1 32 0.832

F3 1 3 0.880

F3 1 5 0.904

F3 1 10 0.904

F3 1 18 0.880

F3 1 31 0.904

F3 1 32 0.952

M3 1 4 0.928

M3 1 6 0.880

M3 1 6 0.928

M3 1 18 0.832

M3 1 20 0.856

M3 1 31 0.904

M4 1 2 0.832

M4 1 2 0.880

M4 1 24 0.928

M4 1 24 0.928

M4 1 28 0.784

M4 1 30 0.976

F1 2 5 0.760

F1 2 13 0.736

F1 2 16 0.640

F1 2 27 0.688

F1 2 31 0.616

F1 2 31 0.856

F3 2 1 0.832

F3 2 22 0.760

F3 2 26 0.736

Talker Condition Listener RA

F3 2 28 0.688

F3 2 30 0.784

F3 2 31 0.760

M3 2 6 0.856

M3 2 11 0.520

M3 2 17 0.664

M3 2 26 0.808

M3 2 28 0.664

M3 2 32 0.664

M4 2 1 0.952

M4 2 5 0.904

M4 2 10 0.760

M4 2 10 0.784

M4 2 13 0.808

M4 2 31 0.856

F1 3 8 0.952

F1 3 14 0.904

F1 3 14 0.736

F1 3 15 0.880

F1 3 16 0.688

F1 3 17 0.952

F3 3 1 0.832

F3 3 1 0.832

F3 3 2 0.904

F3 3 4 0.712

F3 3 6 0.760

F3 3 15 0.712

M3 3 1 0.952

M3 3 2 0.832

M3 3 8 0.784

M3 3 9 0.832

M3 3 11 0.952

M3 3 14 0.928

Talker Condition Listener RA

M4 3 1 0.880

M4 3 4 0.856

M4 3 6 0.712

M4 3 9 0.904

M4 3 13 0.808

M4 3 20 0.856

F1 4 1 0.760

F1 4 10 0.712

F1 4 14 0.592

F1 4 21 0.712

F1 4 23 0.736

F1 4 31 0.616

F3 4 6 0.712

F3 4 12 0.616

F3 4 15 0.520

F3 4 16 0.448

F3 4 25 0.616

F3 4 26 0.640

M3 4 4 0.664

M3 4 4 0.448

M3 4 17 0.568

M3 4 23 0.496

M3 4 29 0.424

M3 4 32 0.520

M4 4 9 0.640

M4 4 14 0.568

M4 4 16 0.568

M4 4 25 0.640

M4 4 27 0.688

M4 4 32 0.640

F1 5 7 0.520

F1 5 10 0.592

F1 5 22 0.592
107

Talker Condition Listener RA

F1 5 23 0.640

F1 5 24 0.664

F1 5 25 0.616

F3 5 4 0.520

F3 5 5 0.568

F3 5 11 0.592

F3 5 16 0.376

F3 5 19 0.544

F3 5 31 0.640

M3 5 2 0.448

M3 5 4 0.544

M3 5 6 0.544

M3 5 11 0.280

M3 5 12 0.448

M3 5 17 0.448

M4 5 6 0.544

M4 5 7 0.520

M4 5 16 0.520

M4 5 22 0.760

M4 5 30 0.688

M4 5 30 0.688

F1 6 1 0.832

F1 6 7 0.856

F1 6 9 0.856

F1 6 13 0.760

F1 6 14 0.760

F1 6 20 0.784

F3 6 4 0.688

F3 6 8 0.904

F3 6 9 0.880

F3 6 10 0.616

F3 6 16 0.760

F3 6 16 0.736

M3 6 5 0.784

M3 6 7 0.856

M3 6 8 0.880

M3 6 11 0.688

M3 6 19 0.760

M3 6 20 0.760

M4 6 1 0.880

M4 6 6 0.784

Talker Condition Listener RA

M4 6 9 0.880

M4 6 12 0.784

M4 6 16 0.880

M4 6 19 0.832

F1 7 5 0.880

F1 7 6 0.760

F1 7 14 0.928

F1 7 14 0.928

F1 7 15 0.832

F1 7 15 0.808

F3 7 3 0.760

F3 7 4 0.856

F3 7 7 0.904

F3 7 11 0.856

F3 7 12 0.736

F3 7 20 0.688

M3 7 4 0.832

M3 7 6 0.856

M3 7 9 0.808

M3 7 14 0.856

M3 7 16 0.640

M3 7 20 0.808

M4 7 3 0.592

M4 7 4 0.928

M4 7 12 0.880

M4 7 14 0.760

M4 7 15 0.856

M4 7 20 0.856

F1 8 4 0.640

F1 8 10 0.568

F1 8 12 0.808

F1 8 25 0.760

F1 8 25 0.712

F1 8 30 0.808

F3 8 2 0.784

F3 8 2 0.760

F3 8 18 0.760

F3 8 19 0.736

F3 8 26 0.760

F3 8 31 0.616

M3 8 2 0.496

Talker Condition Listener RA

M3 8 10 0.568

M3 8 16 0.568

M3 8 20 0.520

M3 8 25 0.544

M3 8 27 0.496

M4 8 3 0.712

M4 8 3 0.808

M4 8 4 0.568

M4 8 17 0.784

M4 8 26 0.784

M4 8 30 0.832

F1 9 1 0.784

F1 9 5 0.712

F1 9 11 0.760

F1 9 15 0.808

F1 9 19 0.712

F1 9 22 0.856

F3 9 5 0.880

F3 9 9 0.688

F3 9 10 0.568

F3 9 17 0.568

F3 9 20 0.736

F3 9 26 0.640

M3 9 1 0.736

M3 9 2 0.664

M3 9 10 0.568

M3 9 10 0.520

M3 9 11 0.688

M3 9 25 0.640

M4 9 17 0.736

M4 9 19 0.736

M4 9 21 0.712

M4 9 23 0.640

M4 9 24 0.760

M4 9 26 0.712

F1 10 3 0.688

F1 10 8 0.544

F1 10 14 0.448

F1 10 15 0.472

F1 10 23 0.424

F1 10 25 0.592
108

Talker Condition Listener RA

F3 10 2 0.304

F3 10 5 0.664

F3 10 18 0.592

F3 10 28 0.544

F3 10 30 0.664

F3 10 32 0.448

M3 10 2 0.376

M3 10 13 0.256

M3 10 16 0.304

M3 10 18 0.328

M3 10 22 0.424

M3 10 26 0.400

M4 10 4 0.544

M4 10 5 0.544

M4 10 11 0.448

M4 10 18 0.592

M4 10 19 0.520

M4 10 24 0.496

F1 11 6 0.736

F1 11 6 0.784

F1 11 11 0.568

F1 11 14 0.712

F1 11 15 0.592

F1 11 18 0.688

F3 11 3 0.688

F3 11 3 0.664

F3 11 6 0.736

F3 11 12 0.616

F3 11 17 0.592

F3 11 20 0.520

M3 11 1 0.448

M3 11 4 0.544

M3 11 8 0.520

M3 11 12 0.472

M3 11 14 0.544

M3 11 19 0.592

M4 11 3 0.640

M4 11 7 0.688

M4 11 10 0.592

M4 11 14 0.712

M4 11 18 0.640

Talker Condition Listener RA

M4 11 20 0.448

F1 12 4 0.376

F1 12 5 0.520

F1 12 6 0.448

F1 12 9 0.448

F1 12 17 0.352

F1 12 29 0.304

F3 12 5 0.328

F3 12 11 0.304

F3 12 18 0.520

F3 12 19 0.376

F3 12 20 0.448

F3 12 28 0.328

M3 12 3 0.280

M3 12 4 0.232

M3 12 11 0.208

M3 12 20 0.304

M3 12 21 0.232

M3 12 25 0.280

M4 12 8 0.280

M4 12 19 0.280

M4 12 24 0.280

M4 12 24 0.448

M4 12 28 0.256

M4 12 31 0.304

F1 13 5 0.736

F1 13 6 0.784

F1 13 8 0.808

F1 13 12 0.544

F1 13 17 0.616

F1 13 19 0.544

F3 13 6 0.544

F3 13 8 0.592

F3 13 10 0.616

F3 13 12 0.664

F3 13 13 0.640

F3 13 17 0.592

M3 13 3 0.616

M3 13 4 0.592

M3 13 6 0.688

M3 13 9 0.520

Talker Condition Listener RA

M3 13 10 0.448

M3 13 13 0.424

M4 13 5 0.616

M4 13 5 0.736

M4 13 6 0.664

M4 13 8 0.712

M4 13 12 0.592

M4 13 13 0.664

F1 14 4 0.568

F1 14 11 0.496

F1 14 12 0.520

F1 14 26 0.640

F1 14 29 0.664

F1 14 31 0.712

F3 14 3 0.760

F3 14 4 0.352

F3 14 8 0.520

F3 14 12 0.616

F3 14 13 0.592

F3 14 31 0.736

M3 14 4 0.376

M3 14 6 0.424

M3 14 8 0.376

M3 14 16 0.496

M3 14 20 0.424

M3 14 26 0.496

M4 14 3 0.664

M4 14 4 0.472

M4 14 8 0.568

M4 14 10 0.592

M4 14 14 0.400

M4 14 17 0.544

F1 15 2 0.880

F1 15 8 0.928

F1 15 12 0.904

F1 15 15 0.856

F1 15 19 0.856

F1 15 27 0.952

F3 15 1 0.904

F3 15 2 0.952

F3 15 6 0.904
109

Talker Condition Listener RA

F3 15 22 0.880

F3 15 22 0.976

F3 15 24 0.904

M3 15 4 0.760

M3 15 10 0.904

M3 15 19 0.952

M3 15 26 0.880

M3 15 30 0.952

M3 15 31 0.856

M4 15 4 0.856

M4 15 15 0.928

M4 15 15 0.952

M4 15 21 0.976

M4 15 23 0.904

M4 15 25 0.952

F1 16 6 0.736

F1 16 10 0.592

F1 16 18 0.808

F1 16 23 0.808

F1 16 25 0.784

F1 16 26 0.688

F3 16 2 0.736

F3 16 6 0.832

F3 16 12 0.784

F3 16 21 0.928

F3 16 26 0.784

F3 16 30 0.784

M3 16 7 0.664

M3 16 9 0.760

M3 16 16 0.856

M3 16 19 0.736

M3 16 26 0.784

M3 16 26 0.784

M4 16 1 0.784

M4 16 9 0.760

M4 16 13 0.760

M4 16 14 0.712

M4 16 15 0.760

M4 16 22 0.880

F1 17 4 0.904

F1 17 7 0.808

Talker Condition Listener RA

F1 17 10 0.784

F1 17 12 0.832

F1 17 14 0.904

F3 17 4 0.736

F3 17 6 0.808

F3 17 7 0.760

F3 17 11 0.760

F3 17 13 0.856

F3 17 16 0.856

M3 17 1 0.808

M3 17 4 0.784

M3 17 4 0.928

M3 17 6 0.808

M3 17 9 0.832

M3 17 9 0.880

M4 17 2 0.856

M4 17 10 0.808

M4 17 13 0.928

M4 17 13 0.904

M4 17 15 0.808

M4 17 16 0.760

F1 17 4 0.904

F1 18 7 0.616

F1 18 11 0.496

F1 18 18 0.664

F1 18 21 0.736

F1 18 29 0.616

F1 18 31 0.784

F3 18 7 0.496

F3 18 8 0.784

F3 18 10 0.640

F3 18 15 0.736

F3 18 20 0.472

F3 18 28 0.616

M3 18 3 0.496

M3 18 4 0.544

M3 18 5 0.424

M3 18 5 0.520

M3 18 9 0.328

M3 18 20 0.520

M4 18 5 0.664

Talker Condition Listener RA

M4 18 6 0.592

M4 18 7 0.568

M4 18 8 0.592

M4 18 11 0.520

M4 18 11 0.640

F1 19 4 0.592

F1 19 6 0.664

F1 19 8 0.520

F1 19 8 0.760

F1 19 10 0.568

F1 19 14 0.592

F3 19 6 0.568

F3 19 14 0.496

F3 19 14 0.496

F3 19 30 0.736

F3 19 30 0.568

F3 19 32 0.592

M3 19 7 0.352

M3 19 7 0.376

M3 19 18 0.568

M3 19 20 0.472

M3 19 23 0.376

M3 19 29 0.496

M4 19 3 0.400

M4 19 6 0.616

M4 19 10 0.544

M4 19 18 0.616

M4 19 23 0.568

M4 19 30 0.664

F1 20 2 0.784

F1 20 5 0.904

F1 20 9 0.928

F1 20 10 0.808

F1 20 14 0.832

F1 20 15 0.880

F3 20 2 0.832

F3 20 3 0.904

F3 20 11 0.832

F3 20 17 0.880

F3 20 19 0.736

F3 20 20 0.808
110

Talker Condition Listener RA

M3 20 2 0.712

M3 20 7 0.856

M3 20 8 0.808

M3 20 11 0.856

M3 20 13 0.832

M3 20 14 0.688

M4 20 2 0.832

M4 20 7 0.808

M4 20 12 0.856

M4 20 16 0.880

M4 20 18 0.760

M4 20 20 0.712

F1 21 1 0.832

F1 21 2 0.832

F1 21 16 0.832

F1 21 17 0.856

F1 21 19 0.736

F1 21 20 0.856

F3 21 2 0.736

F3 21 6 0.928

F3 21 8 0.856

F3 21 9 0.784

F3 21 16 0.832

F3 21 18 0.784

M3 21 2 0.784

M3 21 3 0.808

M3 21 6 0.880

M3 21 9 0.832

M3 21 17 0.736

M3 21 20 0.712

M4 21 6 0.976

M4 21 9 0.808

M4 21 10 0.664

M4 21 11 0.712

M4 21 11 0.952

M4 21 19 0.880

F1 22 8 0.664

F1 22 16 0.664

F1 22 17 0.640

F1 22 21 0.784

F1 22 23 0.736

Talker Condition Listener RA

F1 22 26 0.736

F3 22 1 0.616

F3 22 12 0.664

F3 22 13 0.568

F3 22 29 0.640

F3 22 30 0.856

F3 22 32 0.664

M3 22 12 0.616

M3 22 14 0.424

M3 22 14 0.544

M3 22 18 0.640

M3 22 24 0.760

M3 22 31 0.448

M4 22 1 0.664

M4 22 5 0.808

M4 22 10 0.664

M4 22 11 0.664

M4 22 14 0.616

M4 22 21 0.664

F1 23 3 0.664

F1 23 5 0.808

F1 23 9 0.544

F1 23 11 0.664

F1 23 23 0.712

F1 23 30 0.736

F3 23 3 0.736

F3 23 16 0.568

F3 23 22 0.784

F3 23 24 0.544

F3 23 24 0.688

F3 23 26 0.688

M3 23 3 0.424

M3 23 8 0.472

M3 23 15 0.448

M3 23 25 0.448

M3 23 25 0.616

M3 23 31 0.472

M4 23 4 0.424

M4 23 8 0.496

M4 23 15 0.616

M4 23 22 0.592

Talker Condition Listener RA

M4 23 24 0.448

M4 23 24 0.568

F1 24 1 0.424

F1 24 7 0.520

F1 24 11 0.448

F1 24 24 0.496

F1 24 28 0.328

F1 24 29 0.472

F3 24 16 0.568

F3 24 17 0.400

F3 24 20 0.448

F3 24 20 0.400

F3 24 25 0.664

F3 24 30 0.472

M3 24 1 0.208

M3 24 6 0.424

M3 24 6 0.328

M3 24 21 0.208

M3 24 27 0.376

M3 24 29 0.304

M4 24 6 0.448

M4 24 17 0.256

M4 24 17 0.496

M4 24 22 0.520

M4 24 23 0.304

M4 24 31 0.304

F1 25 3 0.688

F1 25 3 0.568

F1 25 4 0.520

F1 25 6 0.688

F1 25 11 0.616

F1 25 18 0.688

F3 25 3 0.592

F3 25 8 0.568

F3 25 12 0.568

F3 25 12 0.568

F3 25 13 0.424

F3 25 18 0.616

M3 25 5 0.424

M3 25 6 0.424

M3 25 13 0.520
111

Talker Condition Listener RA

M3 25 15 0.448

M3 25 18 0.304

M3 25 20 0.472

M4 25 2 0.520

M4 25 5 0.568

M4 25 5 0.712

M4 25 14 0.448

M4 25 15 0.400

M4 25 15 0.664

F1 26 2 0.520

F1 26 2 0.400

F1 26 2 0.376

F1 26 17 0.304

F1 26 17 0.304

F1 26 23 0.304

F3 26 3 0.328

F3 26 5 0.376

F3 26 22 0.424

F3 26 24 0.280

F3 26 27 0.328

F3 26 27 0.232

M3 26 6 0.376

M3 26 21 0.256

M3 26 25 0.256

M3 26 26 0.208

M3 26 30 0.400

M3 26 30 0.328

M4 26 2 0.184

M4 26 8 0.280

M4 26 9 0.232

M4 26 15 0.208

M4 26 18 0.544

M4 26 25 0.424

F1 27 2 0.568

F1 27 3 0.640

F1 27 4 0.640

F1 27 5 0.592

F1 27 9 0.664

F1 27 10 0.376

F3 27 1 0.544

F3 27 5 0.664

Talker Condition Listener RA

F3 27 9 0.640

F3 27 11 0.664

F3 27 18 0.376

F3 27 19 0.496

M3 27 1 0.472

M3 27 7 0.544

M3 27 13 0.616

M3 27 15 0.376

M3 27 17 0.400

M3 27 17 0.352

M4 27 1 0.616

M4 27 8 0.640

M4 27 9 0.448

M4 27 10 0.472

M4 27 10 0.520

M4 27 19 0.424

F1 28 2 0.664

F1 28 3 0.520

F1 28 13 0.472

F1 28 13 0.592

F1 28 17 0.568

F1 28 22 0.664

F3 28 1 0.592

F3 28 6 0.592

F3 28 10 0.448

F3 28 14 0.424

F3 28 21 0.520

F3 28 32 0.472

M3 28 2 0.256

M3 28 6 0.232

M3 28 15 0.376

M3 28 20 0.376

M3 28 28 0.256

M3 28 31 0.352

M4 28 8 0.568

M4 28 12 0.544

M4 28 13 0.400

M4 28 19 0.376

M4 28 21 0.592

M4 28 32 0.424

F1 29 7 0.904

Talker Condition Listener RA

F1 29 11 0.928

F1 29 12 0.904

F1 29 16 0.928

F1 29 31 0.832

F1 29 31 0.904

F3 29 2 0.808

F3 29 9 0.856

F3 29 13 0.904

F3 29 18 0.856

F3 29 25 0.808

F3 29 30 0.856

M3 29 1 0.784

M3 29 7 0.736

M3 29 13 0.832

M3 29 14 0.928

M3 29 24 0.832

M3 29 30 0.760

M4 29 6 0.976

M4 29 9 0.784

M4 29 18 0.928

M4 29 24 0.952

M4 29 28 0.928

M4 29 31 0.856

F1 30 6 0.664

F1 30 10 0.760

F1 30 13 0.520

F1 30 22 0.688

F1 30 24 0.640

F1 30 32 0.640

F3 30 4 0.640

F3 30 6 0.808

F3 30 18 0.856

F3 30 22 0.712

F3 30 24 0.784

F3 30 32 0.568

M3 30 1 0.712

M3 30 3 0.736

M3 30 7 0.616

M3 30 8 0.592

M3 30 13 0.568

M3 30 20 0.544
112

Talker Condition Listener RA

M4 30 2 0.784

M4 30 12 0.832

M4 30 13 0.688

M4 30 20 0.784

M4 30 21 0.856

M4 30 24 0.616

F1 31 1 0.712

F1 31 5 0.808

F1 31 8 0.856

F1 31 10 0.712

F1 31 12 0.592

F1 31 17 0.736

F3 31 1 0.472

F3 31 1 0.736

F3 31 4 0.640

F3 31 13 0.760

F3 31 14 0.616

F3 31 15 0.712

M3 31 4 0.640

M3 31 5 0.712

M3 31 6 0.616

M3 31 14 0.736

M3 31 17 0.688

M3 31 19 0.664

M4 31 2 0.808

M4 31 6 0.784

M4 31 9 0.808

M4 31 12 0.808

M4 31 12 0.784

M4 31 15 0.688

F1 32 4 0.520

F1 32 7 0.496

F1 32 9 0.496

F1 32 14 0.448

F1 32 16 0.544

F1 32 20 0.472

F3 32 8 0.472

F3 32 9 0.352

F3 32 10 0.472

F3 32 12 0.592

F3 32 19 0.616

Talker Condition Listener RA

F3 32 20 0.544

M3 32 1 0.352

M3 32 3 0.520

M3 32 17 0.616

M3 32 26 0.448

M3 32 27 0.520

M3 32 32 0.376

M4 32 1 0.448

M4 32 2 0.376

M4 32 22 0.640

M4 32 27 0.496

M4 32 31 0.616

M4 32 32 0.544

F1 33 1 0.448

F1 33 11 0.376

F1 33 19 0.496

F1 33 24 0.616

F1 33 28 0.400

F1 33 32 0.448

F3 33 4 0.448

F3 33 9 0.472

F3 33 20 0.400

F3 33 27 0.520

F3 33 29 0.592

F3 33 30 0.712

M3 33 5 0.472

M3 33 12 0.424

M3 33 17 0.424

M3 33 20 0.544

M3 33 29 0.232

M3 33 29 0.472

M4 33 1 0.568

M4 33 4 0.544

M4 33 7 0.520

M4 33 8 0.376

M4 33 19 0.640

M4 33 31 0.496

F1 34 4 0.856

F1 34 7 0.832

F1 34 12 0.784

F1 34 14 0.736

Talker Condition Listener RA

F1 34 15 0.616

F1 34 16 0.688

F3 34 2 0.616

F3 34 4 0.640

F3 34 8 0.760

F3 34 13 0.760

F3 34 16 0.664

F3 34 19 0.616

M3 34 5 0.616

M3 34 13 0.712

M3 34 14 0.640

M3 34 14 0.736

M3 34 15 0.688

M3 34 20 0.640

M4 34 3 0.616

M4 34 5 0.856

M4 34 10 0.712

M4 34 10 0.712

M4 34 13 0.664

M4 34 16 0.808

F1 35 7 0.760

F1 35 9 0.664

F1 35 9 0.784

F1 35 10 0.760

F1 35 11 0.688

F1 35 11 0.736

F3 35 8 0.784

F3 35 11 0.712

F3 35 13 0.736

F3 35 16 0.664

F3 35 17 0.592

F3 35 17 0.688

M3 35 6 0.688

M3 35 7 0.784

M3 35 8 0.664

M3 35 14 0.784

M3 35 17 0.736

M3 35 18 0.736

M4 35 5 0.736

M4 35 12 0.688

M4 35 12 0.592
113

Talker Condition Listener RA

M4 35 13 0.688

M4 35 18 0.736

M4 35 19 0.760

F1 36 3 0.616

F1 36 5 0.736

F1 36 9 0.640

F1 36 13 0.472

F1 36 26 0.664

F1 36 31 0.616

F3 36 10 0.592

F3 36 14 0.544

F3 36 15 0.256

F3 36 17 0.520

F3 36 17 0.664

F3 36 29 0.520

M3 36 8 0.328

M3 36 12 0.472

M3 36 14 0.184

M3 36 15 0.448

M3 36 28 0.352

M3 36 29 0.256

M4 36 1 0.376

M4 36 5 0.544

M4 36 10 0.376

M4 36 22 0.544

M4 36 24 0.544

M4 36 26 0.592

F1 37 8 0.856

F1 37 10 0.832

F1 37 21 0.904

F1 37 23 0.784

F1 37 28 0.808

F1 37 31 0.952

F3 37 2 0.928

F3 37 8 0.904

F3 37 9 0.928

F3 37 22 0.904

F3 37 23 0.856

F3 37 23 0.832

M3 37 1 0.760

M3 37 8 0.832

Talker Condition Listener RA

M3 37 13 0.856

M3 37 23 0.952

M3 37 26 0.880

M3 37 30 0.928

M4 37 8 0.904

M4 37 9 0.856

M4 37 16 0.880

M4 37 20 0.856

M4 37 22 0.880

M4 37 24 0.952

F1 38 1 0.616

F1 38 11 0.568

F1 38 20 0.688

F1 38 23 0.640

F1 38 26 0.760

F1 38 26 0.736

F3 38 1 0.880

F3 38 12 0.736

F3 38 14 0.712

F3 38 15 0.640

F3 38 15 0.712

F3 38 30 0.808

M3 38 2 0.664

M3 38 4 0.568

M3 38 6 0.760

M3 38 25 0.736

M3 38 27 0.832

M3 38 31 0.616

M4 38 4 0.664

M4 38 4 0.592

M4 38 7 0.688

M4 38 7 0.712

M4 38 16 0.664

M4 38 29 0.664

F1 39 4 0.832

F1 39 8 0.832

F1 39 12 0.736

F1 39 13 0.736

F1 39 18 0.640

F1 39 19 0.640

F3 39 5 0.736

Talker Condition Listener RA

F3 39 5 0.808

F3 39 7 0.784

F3 39 10 0.544

F3 39 13 0.808

F3 39 17 0.712

M3 39 3 0.640

M3 39 6 0.832

M3 39 10 0.688

M3 39 14 0.688

M3 39 19 0.856

M3 39 20 0.592

M4 39 11 0.784

M4 39 12 0.808

M4 39 12 0.760

M4 39 13 0.688

M4 39 14 0.736

M4 39 16 0.592

F1 40 6 0.496

F1 40 8 0.208

F1 40 9 0.640

F1 40 21 0.496

F1 40 25 0.592

F1 40 30 0.424

F3 40 3 0.472

F3 40 10 0.448

F3 40 14 0.496

F3 40 18 0.496

F3 40 19 0.448

F3 40 23 0.520

M3 40 15 0.376

M3 40 16 0.424

M3 40 16 0.376

M3 40 20 0.520

M3 40 27 0.592

M3 40 32 0.424

M4 40 5 0.616

M4 40 13 0.496

M4 40 23 0.568

M4 40 29 0.328

M4 40 30 0.760

M4 40 30 0.496
114

Talker Condition Listener RA

F1 41 2 0.640

F1 41 4 0.880

F1 41 5 0.616

F1 41 5 0.832

F1 41 7 0.640

F1 41 9 0.712

F3 41 4 0.760

F3 41 9 0.760

F3 41 9 0.640

F3 41 18 0.760

F3 41 19 0.688

F3 41 20 0.760

M3 41 1 0.616

M3 41 2 0.712

M3 41 2 0.664

M3 41 5 0.904

M3 41 8 0.688

M3 41 20 0.784

M4 41 7 0.688

M4 41 10 0.688

M4 41 13 0.712

M4 41 17 0.664

M4 41 17 0.784

M4 41 19 0.640

F1 42 9 0.640

F1 42 11 0.640

F1 42 13 0.544

F1 42 22 0.736

F1 42 26 0.640

F1 42 28 0.784

F3 42 9 0.616

F3 42 18 0.544

F3 42 19 0.688

F3 42 20 0.472

F3 42 28 0.544

F3 42 29 0.616

M3 42 11 0.448

M3 42 12 0.424

M3 42 18 0.448

M3 42 20 0.160

M3 42 24 0.472

Talker Condition Listener RA

M3 42 25 0.400

M4 42 11 0.616

M4 42 14 0.400

M4 42 16 0.472

M4 42 22 0.496

M4 42 28 0.472

M4 42 32 0.448

F1 43 4 0.928

F1 43 10 0.760

F1 43 15 0.832

F1 43 21 0.856

F1 43 31 0.856

F1 43 32 0.832

F3 43 17 0.736

F3 43 18 0.784

F3 43 18 0.784

F3 43 21 0.928

F3 43 21 0.904

F3 43 24 0.928

M3 43 1 0.928

M3 43 9 0.880

M3 43 12 0.760

M3 43 12 0.880

M3 43 21 0.952

M3 43 31 0.760

M4 43 2 0.904

M4 43 8 0.784

M4 43 19 0.712

M4 43 20 0.832

M4 43 21 0.832

M4 43 23 0.880

F1 44 2 0.664

F1 44 3 0.688

F1 44 4 0.592

F1 44 7 0.544

F1 44 13 0.616

F1 44 32 0.568

F3 44 12 0.736

F3 44 15 0.664

F3 44 21 0.640

F3 44 25 0.712

Talker Condition Listener RA

F3 44 26 0.808

F3 44 31 0.760

M3 44 8 0.448

M3 44 9 0.688

M3 44 15 0.616

M3 44 23 0.664

M3 44 27 0.664

M3 44 28 0.568

M4 44 5 0.640

M4 44 5 0.808

M4 44 12 0.712

M4 44 19 0.736

M4 44 24 0.856

M4 44 28 0.736

F1 45 1 0.568

F1 45 2 0.664

F1 45 6 0.736

F1 45 12 0.736

F1 45 17 0.784

F1 45 19 0.592

F3 45 5 0.736

F3 45 10 0.688

F3 45 10 0.568

F3 45 12 0.640

F3 45 15 0.688

F3 45 20 0.736

M3 45 1 0.808

M3 45 3 0.784

M3 45 6 0.712

M3 45 7 0.736

M3 45 8 0.688

M3 45 12 0.784

M4 45 5 0.808

M4 45 7 0.760

M4 45 10 0.808

M4 45 15 0.784

M4 45 16 0.784

M4 45 16 0.760

F1 46 2 0.736

F1 46 5 0.496

F1 46 9 0.400
115

Talker Condition Listener RA

F1 46 11 0.400

F1 46 13 0.496

F1 46 20 0.640

F3 46 8 0.424

F3 46 12 0.592

F3 46 14 0.184

F3 46 23 0.400

F3 46 25 0.568

F3 46 25 0.568

M3 46 1 0.472

M3 46 3 0.472

M3 46 9 0.568

M3 46 12 0.544

M3 46 20 0.472

M3 46 27 0.448

M4 46 5 0.520

M4 46 8 0.520

M4 46 9 0.544

M4 46 14 0.520

M4 46 21 0.592

M4 46 32 0.616

F1 47 1 0.424

F1 47 11 0.568

F1 47 12 0.280

F1 47 14 0.280

F1 47 23 0.496

F1 47 24 0.616

F3 47 7 0.448

F3 47 9 0.472

F3 47 14 0.472

F3 47 14 0.328

F3 47 27 0.592

F3 47 29 0.592

M3 47 5 0.448

M3 47 6 0.448

M3 47 11 0.448

M3 47 12 0.424

M3 47 26 0.616

M3 47 29 0.448

M4 47 2 0.736

M4 47 3 0.544

Talker Condition Listener RA

M4 47 11 0.304

M4 47 14 0.400

M4 47 24 0.592

M4 47 30 0.568

F1 48 5 0.568

F1 48 10 0.664

F1 48 11 0.736

F1 48 13 0.736

F1 48 14 0.472

F1 48 16 0.544

F3 48 1 0.520

F3 48 8 0.664

F3 48 8 0.496

F3 48 18 0.664

F3 48 19 0.664

F3 48 20 0.400

M3 48 1 0.664

M3 48 7 0.616

M3 48 8 0.400

M3 48 10 0.712

M3 48 15 0.712

M3 48 16 0.568

M4 48 4 0.688

M4 48 6 0.616

M4 48 9 0.688

M4 48 9 0.616

M4 48 18 0.664

M4 48 19 0.568

F1 49 2 0.640

F1 49 3 0.568

F1 49 6 0.688

F1 49 7 0.640

F1 49 14 0.616

F1 49 18 0.568

F3 49 2 0.712

F3 49 11 0.616

F3 49 14 0.520

F3 49 15 0.688

F3 49 15 0.568

F3 49 16 0.520

M3 49 8 0.688

Talker Condition Listener RA

M3 49 8 0.520

M3 49 17 0.664

M3 49 17 0.256

M3 49 18 0.592

M3 49 18 0.664

M4 49 5 0.784

M4 49 7 0.616

M4 49 9 0.664

M4 49 15 0.616

M4 49 17 0.592

M4 49 20 0.592

F1 50 2 0.616

F1 50 3 0.544

F1 50 4 0.472

F1 50 11 0.640

F1 50 18 0.520

F1 50 26 0.592

F3 50 4 0.472

F3 50 5 0.688

F3 50 6 0.520

F3 50 16 0.448

F3 50 16 0.496

F3 50 23 0.592

M3 50 5 0.304

M3 50 10 0.424

M3 50 16 0.400

M3 50 16 0.256

M3 50 18 0.448

M3 50 21 0.232

M4 50 1 0.616

M4 50 5 0.568

M4 50 12 0.640

M4 50 22 0.448

M4 50 26 0.544

M4 50 26 0.592

F1 51 19 0.760

F1 51 19 0.952

F1 51 23 0.904

F1 51 28 0.784

F1 51 30 0.928

F1 51 32 0.904
116

Talker Condition Listener RA

F3 51 1 0.832

F3 51 20 0.952

F3 51 27 0.952

F3 51 29 0.784

F3 51 29 0.736

F3 51 32 0.904

M3 51 9 0.832

M3 51 10 0.736

M3 51 18 0.856

M3 51 27 0.880

M3 51 31 0.904

M3 51 32 0.784

M4 51 17 0.832

M4 51 19 0.928

M4 51 24 0.880

M4 51 29 0.784

M4 51 29 0.784

M4 51 31 0.856

F1 52 8 0.616

F1 52 16 0.832

F1 52 16 0.664

F1 52 17 0.568

F1 52 22 0.592

F1 52 27 0.616

F3 52 8 0.784

F3 52 19 0.664

F3 52 19 0.712

F3 52 25 0.784

F3 52 28 0.592

F3 52 29 0.640

M3 52 1 0.760

M3 52 13 0.544

M3 52 15 0.712

M3 52 17 0.712

M3 52 20 0.712

M3 52 30 0.664

M4 52 13 0.640

M4 52 19 0.712

M4 52 27 0.640

M4 52 28 0.736

M4 52 28 0.592

Talker Condition Listener RA

M4 52 31 0.808

F1 53 1 0.664

F1 53 5 0.736

F1 53 7 0.784

F1 53 14 0.664

F1 53 18 0.688

F1 53 18 0.688

F3 53 10 0.688

F3 53 11 0.688

F3 53 13 0.712

F3 53 13 0.712

F3 53 15 0.760

F3 53 18 0.760

M3 53 4 0.832

M3 53 11 0.736

M3 53 17 0.712

M3 53 18 0.784

M3 53 19 0.592

M3 53 20 0.736

M4 53 3 0.568

M4 53 5 0.808

M4 53 8 0.880

M4 53 12 0.760

M4 53 16 0.784

M4 53 19 0.784

F1 54 7 0.424

F1 54 7 0.472

F1 54 16 0.616

F1 54 19 0.496

F1 54 28 0.472

F1 54 29 0.544

F3 54 1 0.544

F3 54 2 0.424

F3 54 12 0.472

F3 54 16 0.376

F3 54 20 0.376

F3 54 21 0.376

M3 54 8 0.448

M3 54 13 0.328

M3 54 15 0.496

M3 54 28 0.520

Talker Condition Listener RA

M3 54 30 0.568

M3 54 32 0.400

M4 54 6 0.616

M4 54 18 0.688

M4 54 19 0.400

M4 54 26 0.664

M4 54 29 0.640

M4 54 30 0.592

F1 55 4 0.664

F1 55 5 0.712

F1 55 10 0.616

F1 55 12 0.712

F1 55 17 0.688

F1 55 18 0.688

F3 55 1 0.760

F3 55 4 0.664

F3 55 10 0.448

F3 55 11 0.568

F3 55 15 0.808

F3 55 20 0.784

M3 55 1 0.712

M3 55 2 0.664

M3 55 3 0.664

M3 55 7 0.736

M3 55 12 0.664

M3 55 17 0.760

M4 55 3 0.928

M4 55 3 0.832

M4 55 4 0.808

M4 55 11 0.640

M4 55 11 0.760

M4 55 15 0.784

F1 56 3 0.664

F1 56 4 0.448

F1 56 10 0.520

F1 56 16 0.448

F1 56 28 0.592

F1 56 29 0.664

F3 56 7 0.376

F3 56 7 0.592

F3 56 13 0.424
117

Talker Condition Listener RA

F3 56 19 0.424

F3 56 21 0.544

F3 56 31 0.568

M3 56 9 0.328

M3 56 13 0.208

Talker Condition Listener RA

M3 56 16 0.256

M3 56 23 0.328

M3 56 27 0.448

M3 56 29 0.256

M4 56 11 0.592

Talker Condition Listener RA

M4 56 15 0.424

M4 56 18 0.520

M4 56 19 0.520

M4 56 23 0.496

M4 56 28 0.472
118

APPENDIX C: SOURCE SPEECH RECORDING INFORMATION

Some speech material in this test was recorded in the laboratory at the Institute for 

Telecommunication Sciences in Boulder, CO. Recordings were performed in an NC-35-rated 

sound isolation chamber according to the following.

The microphone used in the source recording was a Shure Beta 53A microphone sampled at 

48 kHz/16 bit on a Windows®-based computer using commercially available software. Active 

signal level was normalized to -28 dB below overload using the ITU-T Recommendation P.56 

voltmeter software [16] [17].

The speech material spoken by the talkers was the word list defined in the MRT description of 

[6] in the carrier sentence, “Please select the word ....”
119

APPENDIX D: LISTENING LABORATORY CONFIGURATION

Two test chambers were set up to meet the standards set forth in subclauses 8.10.4.10–8.10.4.15 

of [10] with the overall level modification as agreed to by the Audio Performance Working 

Group (APWG). The physical layout of the chambers is shown schematically in Figure D-1. The 

loudspeaker carrying the speech signal sat on a table and was placed equidistant from the 

chamber side walls, on the edge of the table nearest the listener. The listening position was also 

equidistant from the chamber side walls, and 150 cm away from the speech loudspeaker (in 

analogy to the talker-listener distance specified in subclause 8.10.4.10). The two loudspeakers on 

either side of the table were used to produce the pink noise (allowed by subclause 8.10.4.14). 

The loudspeakers were pointed toward the “back” of the room, and were not pointed directly at 

the listener, thus fulfilling subclause 8.10.4.12. The combination of using two loudspeakers to 

produce the pink noise and the distance from the loudspeakers to the listening position created a 

quasi-uniform field of sound, thus satisfying subclause 8.10.4.13. 

For Chamber 1, in order to generate a field of “broadband pink noise” in compliance with 

subclause 8.10.4.11 of [10], a Gold Line Model PN2 Pink Noise Generator (PNG) was used. The 

PNG was modified to accept an external power source so any possible effect of a non-constant 

battery voltage could be avoided. The output of the PNG was fed into a General Radio Model 

1952 Universal Filter, which was tuned to have a bandpass characteristic for the interval between 

400 Hz and 4 kHz. The signal then passed through a mixer, equalizer, and power amplifier and 

was finally delivered to the pair of loudspeakers located in Chamber 1. For Chamber 2, the 

“broadband pink noise” field was created by using a recording of the “broadband pink noise” 

obtained in [6] and played from a CD player. The signal passed through a mixer, equalizer, and 

power amplifier and was finally delivered to the pair of loudspeakers in Chamber 2. A Gold Line 

Model DSP30RM realtime spectrum analyzer was used to analyze the resulting acoustic pink 

noise spectrum in each chamber. Using information from the spectrum analyzer, the equalizers 

were tuned to achieve the best possible response for each test chamber. 

Subclause 8.10.4.11 specifies that the pink noise generated have a tolerance of 6 dB per octave 

band in the range of frequencies between 400 Hz and 4 kHz. Figure D-2 shows the 1/3 octave 

power levels in Chamber 1 and Figure D-3 shows the power levels in Chamber 2. Figure D-2 for 

Chamber 1 shows that the power level varies from -10 dB to 0 dB between 400 Hz and 4 kHz 

(note that the 5 dB per interval light is on). Figure D-3 shows that the power level varies between 

0 dB and +3 dB between 400 Hz and 4 kHz (note that the 3 dB per interval light is on). Both of 

these rooms meet the ±6 dB specification of subclause 8.10.4.11, although Chamber 2 meets it 

more loosely than Chamber 1. This is not expected to impact results as only 11 of the 52 listeners 

(approximately 21%) participated via Chamber 1.
120

Figure D-1. Diagram showing physical layout of test chambers.

A Brüel and Kjær Model 2250 sound level meter, compliant with [20] and [21], was used to 

verify that the noise level met specifications. The noise level in Chamber 1 was measured to be 

64.9 dBA, and the noise level in Chamber 2 was measured to be 65.0 dBA, fulfilling the 

modified level from 8.10.4.15, as agreed to by the APWG.

Speech was generated in the two chambers using Fostex 6301B loudspeakers. The speech signal 

originated in MATLAB, propagated through the PC’s sound card, then to a mixer, and then 

finally to the loudspeaker. When the signal path was active but no signal was being sent to the 

speakers, the noise level in Chamber 1 fell to 21.4 dBA, and the level in Chamber 2 fell to 

21.9 dBA. Undistorted, noise-free speech registered 84.0 dBA in Chamber 1 and 83.8 dBA in 

Chamber 2.
121

Figure D-2. Real time spectrum analysis of pink noise in Chamber 1.

Figure D-3. Real time spectrum analysis of pink noise in Chamber 2.
122

APPENDIX E: COMPARISON OF TALKER VERSUS A HEAD AND TORSO 

SIMULATOR FOR INTELLIGIBILITY TESTING

Because of potential differences between test outcomes when using a live talker wearing an 

SCBA mask and outcomes when using a head and torso simulator (HATS), particularly with 

respect to exhalation noise and vocal tract loading under the positive air pressure of the mask, a 

small experiment was designed and conducted to validate the testing method using a HATS. The 

experiment consisted of two scenarios: wearing an SCBA mask with no background noise and 

wearing an SCBA mask with PASS1 background noise. The scenarios were repeated with a live 

talker and also using the HATS for a total of four conditions. 

The “Talker” conditions were recorded with a talker speaking the MRT word list and holding a 

microphone while wearing a mask, with and without background noise present. The HATS 

conditions used recordings of the same talker, but played through the HATS as indicated in 

Section 2. The recordings were then played for listeners as indicated in Sections 2 and 5. The 

recordings were not passed through the reference systems, in order to keep this experiment as 

simple as possible.

The listening panel consisted of five listeners. They listened to six sessions of 200 sentences 

each, for a total of 1200 results per listener, 6000 results total.

Table E-1 contains the intelligibility scores for this comparison. Figure E-1 contains the bar chart 

of results for this environment. ANOVA showed no significant difference between a talker and a 

HATS. One can therefore infer that the use of the HATS is a reasonable and safe approximation 

of putting a real talker into a high noise environment, and can be expected to yield equivalent 

results.

Table E-1. Intelligibility scores for talker versus HATS

Scenario Talker HATS

Mask w/ no background noise 0.850 0.864

Mask with PASS1 0.852 0.889
123

Figure E-1. Bar chart of talker vs. HATS intelligibility scores.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Mask (Vox Port) Mask w/ PASS

Intelligibility Score

System

Comparing Talker vs. HATS

HATS

Talker
FORM NTIA-29 U.S. DEPARTMENT OF COMMERCE 

(4-80) NAT’L. TELECOMMUNICATIONS AND INFORMATION ADMINISTRATION

BIBLIOGRAPHIC DATA SHEET

1. PUBLICATION NO.

TR-13-495

2. Government Accession No. 3. Recipient’s Accession No.

4. TITLE AND SUBTITLE

Intelligibility of Analog FM and Updated P25 Radio Systems in the Presence of 

Fireground Noise: Test Plan and Results

5. Publication Date

May 2013

6. Performing Organization Code

ITS.P

7. AUTHOR(S)

David J. Atkinson and Andrew A. Catellier

9. Project/Task/Work Unit No.

6692000-320

8. PERFORMING ORGANIZATION NAME AND ADDRESS

Institute for Telecommunication Sciences

National Telecommunications & Information Administration

U.S. Department of Commerce

325 Broadway

Boulder, CO 80305

10. Contract/Grant Number.

11. Sponsoring Organization Name and Address

National Telecommunications & Information Administration

Herbert C. Hoover Building

14th & Constitution Ave., NW

Washington, DC 20230

12. Type of Report and Period Covered

14. SUPPLEMENTARY NOTES

15. ABSTRACT (A 200-word or less factual summary of most significant information. If document includes a significant bibliography or 

literature survey, mention it here.)

This report describes a modified rhyme test (MRT) conducted to characterize the behavior of digital and analog 

communication in the presence of background noise and moderate RF channel degradation. This is done through the use of 

reference systems to provide a manufacturer-independent perspective on this issue.

16. Key Words (Alphabetical order, separated by semicolons)

intelligibility; Project 25; vocoder; modified rhyme test; noise; analog FM; land mobile radio; LMR; public safety; fire 

service

17. AVAILABILITY STATEMENT

UNLIMITED.

FOR OFFICIAL DISTRIBUTION.

18. Security Class. (This report)

Unclassified

20. Number of pages

144

19. Security Class. (This page)

Unclassified

21. Price:
NTIA FORMAL PUBLICATION SERIES

NTIA MONOGRAPH (MG) 

A scholarly, professionally oriented publication dealing with state-of-the-art research or an 

authoritative treatment of a broad area. Expected to have long-lasting value.

NTIA SPECIAL PUBLICATION (SP)

Conference proceedings, bibliographies, selected speeches, course and instructional materials, 

directories, and major studies mandated by Congress.

NTIA REPORT (TR)

Important contributions to existing knowledge of less breadth than a monograph, such as results 

of completed projects and major activities. Subsets of this series include:

NTIA RESTRICTED REPORT (RR)

Contributions that are limited in distribution because of national security classification or 

Departmental constraints. 

NTIA CONTRACTOR REPORT (CR)

Information generated under an NTIA contract or grant, written by the contractor, and 

considered an important contribution to existing knowledge.

JOINT NTIA/OTHER-AGENCY REPORT (JR)

This report receives both local NTIA and other agency review. Both agencies’ logos and 

report series numbering appear on the cover. 

NTIA SOFTWARE & DATA PRODUCTS (SD)

Software such as programs, test data, and sound/video files. This series can be used to transfer 

technology to U.S. industry.

NTIA HANDBOOK (HB)

Information pertaining to technical procedures, reference and data guides, and formal user's 

manuals that are expected to be pertinent for a long time.

NTIA TECHNICAL MEMORANDUM (TM)

Technical information typically of less breadth than an NTIA Report. The series includes data, 

preliminary project results, and information for a specific, limited audience.

For information about NTIA publications, contact the NTIA/ITS Technical Publications Office at 325 

Broadway, Boulder, CO, 80305 Tel. (303) 497-3572 or e-mail info@its.bldrdoc.gov. 

This report is for sale by the National Technical Information Service, 5285 Port Royal Road, Springfield, 

VA 22161,Tel. (800) 553-6847.