U.S. DEPARTMENT OF COMMERCE • National Telecommunications and Information Administration

report series

NTIA Report 15-517

3.5 GHz Exclusion Zone 

Analyses and Methodology

E. Drocella

J. Richards

R. Sole

F. Najmy

A. Lundy

P. McKenna

U.S. DEPARTMENT OF COMMERCE

June 2015

Reissued March 2016

NTIA Report 15-517

3.5 GHz Exclusion Zone 

Analyses and Methodology

E. Drocella

J. Richards

R. Sole

F. Najmy

A. Lundy

P. McKenna

iii

DISCLAIMER

Certain commercial equipment and materials are identified in this report to specify adequately 

the technical aspects of the reported results. In no case does such identification imply 

recommendation or endorsement by the National Telecommunications and Information

Administration, nor does it imply that the material or equipment identified is the best available 

for this purpose.

v

PREFACE

The National Telecommunications and Information Administration’s Office of Spectrum 

Management (OSM) and Institute for Telecommunication Sciences (ITS) would like to 

acknowledge and thank the staff of the Federal Communications Commission and the 

Department of Defense for their hard work and dedication in the preparation of this report that 

will lead to sharing the 3550-3650 MHz band between federal radar operations and commercial 

broadband systems. OSM and ITS would also like to acknowledge the assistance received from 

the Navy and Army spectrum management offices.

NOTE ON REISSUE

This report was reissued in March 2016 to correct a typographical error in (A-21) and to clarify 

certain terms in Sections 4 and 5.

vii

CONTENTS

Preface..............................................................................................................................................v

Note on Reissue ...............................................................................................................................v

Figures............................................................................................................................................ ix

Tables............................................................................................................................................ xii

Abbreviations/Acronyms............................................................................................................. xiii

Executive Summary.......................................................................................................................xv

1. Introduction..................................................................................................................................1

2. CBSD Technical and Deployment Parameters............................................................................3

2.1 Small Cell Device Technical Characteristics.........................................................................3

2.2 Small Cell Device Deployment Characteristics.....................................................................4

3. Radar Technical and Deployment Parameters.............................................................................7

3.1 Radar Technical Parameters ..................................................................................................7

3.2 Radar Deployment Parameters ..............................................................................................8

4. Description of Shipborne Radar Analysis Methodology.............................................................9

4.1 Analysis Overview.................................................................................................................9

4.2 Interference Link Budget.......................................................................................................9

4.3 Radar Receiver Interference Threshold ...............................................................................10

4.4 AP and UE EIRP Values .....................................................................................................11

4.5 Radar Receive Antenna Gain...............................................................................................11

4.6 Transmitter and Receiver Insertion Losses..........................................................................11

4.7 Propagation and Clutter Loss...............................................................................................11

4.8 Building Attenuation Loss...................................................................................................13

4.9 Frequency Dependent Rejection..........................................................................................13

4.10 Shipborne Radar Exclusion Zone Distance Analysis Output ............................................14

5. Description of Ground-Based Radar Analysis Methodology ....................................................18

5.1 Interference Link Budget.....................................................................................................19

5.2 AP EIRP and Antenna Height .............................................................................................20

5.3 Radar Receive Antenna Gain...............................................................................................20

5.4 Basic Transmission Loss .....................................................................................................20

5.5 Clutter Loss..........................................................................................................................20

5.6 FDR......................................................................................................................................20

5.7 Radar Receiver Interference Threshold ...............................................................................21

5.8 Ground-Based Radar Exclusion Zone Distance Analysis Output .......................................21

6. Summary of Exclusion Zone Distance Analysis .......................................................................23
viii

7. References..................................................................................................................................25

Appendix A Description of Propagation and Clutter Loss Model.................................................27

A.1 Introduction.........................................................................................................................27

A.2 Propagation Model Overview.............................................................................................27

A.3 The Extended Hata Model ..................................................................................................28

A.4 The Irregular Terrain Model (ITM)....................................................................................35

A.4.1 The Reference Attenuation........................................................................................39

A.4.2 The Coefficients of the Diffraction Range ................................................................40

A.4.3 The Diffraction Range Attenuation Function............................................................40

A.4.4 The Coefficients of the Line-of-Sight Range ............................................................42

A.4.5 The Line-of-Sight Range Attenuation Function........................................................46

A.4.6 The Coefficients of the Forward (or Tropospheric) Scatter Range ...........................47

A.4.7 The Scatter Range Attenuation Function ..................................................................48

A.4.8 Variability and the Quantiles of the Attenuation.......................................................50

A.5 References...........................................................................................................................54

Appendix B Shipborne Radar Analysis Results ............................................................................56

B.1 Introduction.........................................................................................................................56

B.2 Shipborne Radar Analysis Results......................................................................................56

B.3 References...........................................................................................................................75

Appendix C Ground-Based Radar Analysis Results .....................................................................76

C.1 Introduction.........................................................................................................................76

C.2 Ground-Based Radar Analysis Results...............................................................................76

C.3 References...........................................................................................................................82
ix

FIGURES

Figure 1. Example—Exclusion zone distances referenced to the coastline for 

different confidence intervals. .......................................................................................................15

Figure 2. Example—Exclusion zone distances for different confidence intervals 

for a shipborne radar at S2.............................................................................................................16

Figure 3. Example of Visualyse simulation area. ..........................................................................19

Figure 4. AP emission spectra measurement used for the model and the analysis........................21

Figure 5. Example output–IIII cumulative distribution of interference probability.......................22

Figure B-1. Shipborne Radar 1–exclusion zone lower 48 states (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................56

Figure B-2. Shipborne radar 1–Exclusion zone upper west coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................57

Figure B-3. Shipborne Radar 1–Exclusion zone middle west coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................57

Figure B-4. Shipborne Radar 1–Exclusion zone lower west coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................58

Figure B-5. Shipborne Radar 1–Exclusion zone upper east coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................58

Figure B-6. Shipborne Radar 1–Exclusion zone middle east coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................59

Figure B-7. Shipborne Radar 1–exclusion zone lower east coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................59

Figure B-8. Shipborne Radar 1–Exclusion zone eastern gulf coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................60

Figure B-9. Shipborne Radar 1–Exclusion zone western gulf coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................60

Figure B-10. Shipborne Radar 4–Exclusion zone lower 48 states (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................61

Figure B-11. Shipborne Radar 4–Exclusion zone upper west coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................61
x

Figure B-12. Shipborne Radar 4–Exclusion zone middle west coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................62

Figure B-13. Shipborne Radar 4–Exclusion zone lower west coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................62

Figure B-14. Shipborne Radar 4–Exclusion zone upper east coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................63

Figure B-15. Shipborne Radar 4–Exclusion zone middle east coast(yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................63

Figure B-16. Shipborne Radar 4–exclusion zone lower east coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................64

Figure B-17. Shipborne Radar 4–Exclusion zone eastern gulf coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................64

Figure B-18. Shipborne Radar 4–Exclusion zone eastern gulf coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................65

Figure B-19. Shipborne Radar 5–Exclusion zone lower 48 states(yellow line–fast 

track exclusion zone and blue line–revised exclusion zone). ........................................................65

Figure B-20. Shipborne Radar 5–Exclusion zone upper west coast(yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................66

Figure B-21. Shipborne Radar 5–Exclusion zone middle west coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................66

Figure B-22. Shipborne Radar 5–Exclusion zone lower west coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................67

Figure B-23. Shipborne Radar 5–Exclusion zone upper east coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................67

Figure B-24. Shipborne Radar 5–Exclusion zone middle east coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................68

Figure B-25. Shipborne Radar 5–Exclusion zone lower east coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................68

Figure B-26. Shipborne Radar 5–Exclusion zone eastern gulf coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................69

Figure B-27. Shipborne Radar 5–Exclusion zone western gulf coast (yellow line–

fast track exclusion zone and blue line–revised exclusion zone). .................................................69
xi

Figure B-28. Composite shipborne radar exclusion zone lower 48 states (yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................70

Figure B-29. Composite shipborne radar exclusion zone upper west coast (yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................70

Figure B-30. Composite shipborne radar exclusion zone middle west coast 

(yellow line–fast track exclusion zone and blue line–revised exclusion zone). ............................71

Figure B-31. Composite shipborne radar exclusion zone lower west coast (yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................71

Figure B-32. Composite shipborne radar exclusion zone upper east coast (yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................72

Figure B-33. Composite shipborne radar exclusion zone middle east coast (yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................72

Figure B-34. Composite shipborne radar exclusion zone lower east coast(yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................73

Figure B-35. Composite shipborne radar exclusion zone eastern gulf coast (yellow 

line–fast track exclusion zone and blue line–revised exclusion zone). .........................................73

Figure B-36. Composite shipborne radar exclusion zone western gulf coast 

(yellow line–fast track exclusion zone and blue line–revised exclusion zone). ............................74

Figure C-1. Ground-based radar exclusion zone lower 48 states. .................................................78

Figure C-2. Ground-based radar exclusion zone upper west coast................................................78

Figure C-3. Ground-based radar exclusion zone middle west coast..............................................79

Figure C-4. Ground-based radar exclusion zone upper central. ....................................................79

Figure C-5. Ground-based radar exclusion zone lower central. ....................................................80

Figure C-6. Ground-based radar exclusion zone upper east coast.................................................80

Figure C-7. Ground-based radar exclusion zone middle east coast...............................................81

Figure C-8. Ground-based radar exclusion zone lower east coast.................................................81
xii

TABLES

Table 1. Small Cell Device (CBSD) Technical Characteristics Used in Exclusion 

Zone Analysis ..................................................................................................................................3

Table 2. Example Calculation for Number of APs..........................................................................6

Table 3. Antenna Directivity Parameters.........................................................................................7

Table 4. Peak and Average Mask Pattern Equations.......................................................................7

Table 5. Summary of Shipborne Radar Analysis Results..............................................................23

Table 6. Summary of Shipborne Radar Analysis Results..............................................................23

Table 7. Summary of Shipborne Radar Analysis Results..............................................................23

Table 8. Summary of Ground-Based Radar Analysis Results.......................................................24

Table A-1. Okumura et al. Location Variability Standard Deviations..........................................34

Table A-2. Irregular Terrain Model Input Parameters/Data and Preliminary 

Calculations....................................................................................................................................35

Table A-3. Longley-Rice Low Probability or “Ducting” Constants..............................................53

Table B-1. Summary of Shipborne Radar Analysis Results..........................................................74

Table B-2. Comparison of Summary of Shipborne Radar Analysis Results.................................75

Table B-3. Summary of Shipborne Radar Analysis Results by Area Affected.............................75

Table C-1. Ground-Based Radar 1 Locations................................................................................76

Table C-2. Ground-Based Radar 3 Locations................................................................................77

Table C-3. Results of Ground-Based Radar Analyses...................................................................77
xiii

ABBREVIATIONS/ACRONYMS

AP access point

CBRS Citizens Broadband Radio Service

CBSD Citizens Broadband Radio Service Device

dB decibel

dBi decibels relative to an isotropic antenna

dBm decibels relative to a milliwatt

DoD Department of Defense

EIRP equivalent isotropically radiated power

FCC Federal Communications Commission

FDR frequency dependent rejection

GHz gigahertz

GIS Geographic Information System

II

NN

interference-to-noise ratio (dB)

IF Intermediate Frequency

ITM ITS Irregular Terrain Model

ITS Institute for Telecommunication Sciences

log common logarithm

ln natural logarithm

MHz megahertz

NLCD National Land Cover Database

NTIA National Telecommunications and Information Administration

OFR off-frequency rejection

OTR on-tune rejection

TDD time division duplex

UE user equipment

xv

EXECUTIVE SUMMARY

The National Telecommunications and Information Administration (NTIA) Fast Track 

Report [1] identified the 3550–3650 MHz band (3.5 GHz band) as potentially suitable for 

commercial broadband use. The 3.5 GHz band was one of the candidate bands identified by 

NTIA in response to the President’s initiative [2] to identify 500 megahertz of spectrum for 

commercial wireless broadband. NTIA believes that the 3.5 GHz band is well suited to exploring 

the next generation of shared spectrum technologies, to drive greater productivity and efficiency 

in spectrum use.

The Federal Communications Commission (FCC) voted to approve a rulemaking proceeding in 

GN Docket No. 12-354 [3] to address sharing issues associated with the establishment of a new 

Citizens Broadband Radio Service (CBRS) in the 3.5 GHz band. The Report and Order [4] was 

adopted on April 17, 2015. 

In its Fast Track Report [1], NTIA concluded that large geographic separation and frequency 

offsets could be used to minimize interference between a deployment of commercial high-power 

macro-cell networks seeking access to the 3.5 GHz band and incumbent federal shipborne, 

ground-based, and airborne radar systems that operate or are planned to operate in and adjacent 

to the 3.5 GHz band. In the 3.5 GHz Band Further Notice of Proposed Rulemaking [3], the FCC 

stated that it would work with NTIA to reassess these exclusion zone distances using lowerpowered small cell technology. NTIA, working in collaboration with the FCC and the 

Department of Defense (DoD), performed the analysis to re-evaluate the exclusion zone 

distances needed to protect federal shipborne and ground-based radar systems. This report 

presents the results of this 3.5 GHz Study. This report also provides a description of the technical 

and deployment parameters of Citizens Broadband Radio Service Device (CBSD) access points 

(APs) and user equipment (UE), technical characteristics of federal radar systems, and the 

analysis methodology used to compute the distances that establish revised exclusion zones 

necessary to protect federal radar systems operating in and adjacent to the 3.5 GHz band.

The coastal exclusion zones developed by the 3.5 GHz Study were derived from the composite 

characteristics of existing or planned shipborne radars. The associated maps show the minimum 

separation distances that the devices must be set back from the coastlines so that the aggregate

interference power of the small-cell base stations does not exceed the interference protection 

criteria of the shipboard radar receivers. The UE devices were also analyzed and shown not to be 

of concern in terms of generating interference in the shipboard radar receivers. The updated 

analysis results in a reduction of 77 percent of the total geographic area impacted by the coastal 

exclusion zones (averaged across three shipborne radar systems) compared to the Fast Track 

Report’s findings.

The 3.5 GHz Study also calculated exclusion zones for the ground-based radar systems; the 

resulting protection zones are polygons encompassing selected sites where the ground-based 

radars operate. As a result of the updated analysis, a protection zone of 3 kilometers is required 

around the perimeter boundary of the installations where the ground-based radars are deployed.

The Fast Track Report recommended much larger exclusion zones ranging from 40 to 60 

kilometers.

3.5 GHZ EXCLUSION ZONE ANALYSES AND METHODOLOGY

E. Drocella,J. Richards, R. Sole, F. Najmy, A. Lundy,1 and P. McKenna2

This report describes the 3.5 GHz Study. It explains the assumptions, methods, 

analyses, and system characteristics used to generate the revised exclusion zones 

for small-cell commercial broadband systems to protect federal radar operations 

(ship and land based) from aggregate interference in the band 3550–3650 MHz. 

The 3.5 GHz Study’s exclusion zones are compared with the exclusion zones that 

were generated in the Fast Track Report, which considered macro-cell operations. 

Keywords: NTIA, FCC, radar, exclusion zones, CBRS, CBSD, spectrum sharing, 3550–3650

MHz, 3.5 GHZ band

1. INTRODUCTION

The National Telecommunications and Information Administration (NTIA) Fast Track Report 

[1] identified the 3550–3650 MHz band (3.5 GHz band) as potentially suitable for commercial

broadband use. The 3.5 GHz band is one of the candidate bands identified by NTIA in response 

to the President’s initiative to identify 500 megahertz of spectrum for commercial wireless 

broadband [2]. NTIA believes that the 3.5 GHz band is well suited to exploring the next 

generation of shared spectrum technologies, to drive greater productivity and efficiency in 

spectrum use.

The Federal Communications Commission (FCC) voted to approve a rulemaking proceeding 

under GN Docket No. 12-354 to address sharing issues associated with the establishment of a 

new Citizens Broadband Radio Service (CBRS) in the 3.5 GHz band [3]. The Report and Order

[4], adopted on April 17, 2015, established a roadmap for making the entirety of the 3.5 GHz 

Band available for commercial use in phases. It described a broad, three-tiered sharing 

framework enabled by a Spectrum Access System, adopting a hybrid framework that selects, 

automatically, the best authorization approach based on local supply and demand.

In its Fast Track Report, NTIA concluded that geographic separation and frequency offsets could 

be used to minimize interference between a deployment of commercial high-power macro-cell 

networks seeking access to the 3.5 GHz band and incumbent federal shipborne, ground-based, 

and airborne radar systems that operate or are planned to operate in and adjacent to the 3.5 GHz 

band [1, 1.6]. Based on the Fast Track Report’s analysis, NTIA recommended large exclusion 

zones along U.S. coastlines (East, West, and Gulf) and exclusion zones around selected groundbased radar sites. In the 3.5 GHz Further Notice of Proposed Rulemaking, the FCC stated that it

 1 The first five authors are with the Office of Spectrum Management (OSM), National Telecommunications and 

Information Administration, U.S. Department of Commerce, Washington, D.C. 20230. 2 The author is with the Institute for Telecommunication Sciences (ITS), National Telecommunications and 

Information Administration, U.S. Department of Commerce, Boulder, CO 80305.
2

would work with NTIA to reassess the exclusion zone distances using lower-powered small cell 

technology [3, 31249].

NTIA, working in collaboration with the FCC and the Department of Defense (DoD), reevaluated the exclusion zone distances needed to protect federal shipborne and ground-based 

radar systems. This report describes that study, which is referred to as the 3.5 GHz Study. This 

report describes the technical and deployment parameters of Citizens Broadband Radio Service 

Device (CBSD) access points (APs) and user equipment (UE), technical characteristics of federal 

radar systems, the analysis methodology for computing the revised exclusion zone distances 

necessary to protect federal radar systems operating in and adjacent to the 3.5 GHz band, and the 

results of the exclusion zone distance analysis.
3

2. CBSD TECHNICAL AND DEPLOYMENT PARAMETERS

2.1 Small Cell Device Technical Characteristics 

The 3.5 GHz Study’s exclusion zone distance analysis examined small cell technology for the 

Citizens Band Radio Service Devices (CBSD) technical and deployment parameters. The 

technical parameters for the small cell CBSD APs and UE used in the exclusion zone distance 

analysis are provided in Table 1.

Table 1. Small Cell Device (CBSD) Technical Characteristics Used in Exclusion Zone Analysis 

Parameter Value

Access Points

Transmitted Power to Antenna (dBm) 24 (Outdoor)

20 (Indoor)

Mainbeam Antenna Gain (dBi) 6

Equivalent Isotropically Radiated Power - Outdoor CBSD (dBm) 30

Equivalent Isotropically Radiated Power - Indoor CBSD (dBm) 26

Channel Bandwidth (MHz) 10

Signal Bandwidth (MHz) 9

Channel Usage (%)

Dense Urban/Urban

Suburban 

Rural

60

40

20

Antenna Height (m)

Outdoor 

Indoor - Dense Urban

Indoor - Urban

Indoor - Suburban 

Indoor - Rural 

6

50%: 3 to 15; 25%: 18 to 30; and 25%: 33 to 60

50%: 3 and 50%: 6 to 18

70%: 3 and 30%: 6 to 12

80%: 3 and 20%: 6

Cable, Insertion, or Other Losses (dB) 2 (Outdoor)

User Equipment

Transmitted Power to Antenna (dBm) 24

Mainbeam Antenna Gain (dBi) 0

Equivalent Isotropically Radiated Power (dBm) 24

Channel Bandwidth (MHz) 10

Signal Bandwidth (MHz) 9

Channel Usage (%)

Dense Urban/Urban

Suburban 

Rural

60

40

20

Antenna Height (m) 1.5 (Outdoor)

1.5 below their associated CBSD (Indoor)

The CBSD in the 3.5 GHz band considered in this analysis use a time division duplex (TDD) 

access scheme so the APs and UE will not be transmitting simultaneously. In the exclusion zone 
4

distance analysis the interference effects from APs and UE were computed separately. In 

performing the exclusion zone distance analysis for federal radar systems operating below 3550 

MHz, the measured emission spectrum of a representative TDD microcell base station was used

[5]. 

The maximum allowable equivalent isotropically radiated power (EIRP) level for outdoor APs 

used in the analysis is 30 dBm, which includes a 24 dBm transmitted power to the antenna and 

6 dBi mainbeam antenna gain. In the analysis a fractional transmit power is applied to each AP 

in the dense urban/urban, suburban, and rural regions to account for partial utilization of 

subcarriers (e.g., LTE resource blocks) within a 10 MHz carrier bandwidth. Such partial 

utilization is expected due to traffic dynamics and resource partitioning between small cells to 

reduce intra-system interference. These fractions applied to the outdoor APs are based on the 

channel loading percentages shown in Table 1: 60 percent for dense urban and urban regions, 40 

percent for suburban regions, and 20 percent for rural regions. Using these fractional power 

adjustments the EIRP values for the outdoor APs used in the analysis are: 27.8 dBm for dense 

urban/urban regions; 26 dBm for suburban regions; and 23 dBm for rural regions. For all indoor 

APs the EIRP used in the analysis is 26 dBm based on a 20 dBm transmitted power to the 

antenna and a 6 dBi mainbeam antenna gain. Similar partial loading percentages (60/40/20) are 

also applied to indoor small cells in different regions.

2.2 Small Cell Device Deployment Characteristics 

Geographic Information System (GIS) 2011 National Land Cover Database (NLCD) data and 

overlaid 2010 U.S. Census population data (provided in average population density) were used in 

a multi-step distance analysis process to distribute the small cell APs considered in the exclusion 

zone distance analysis.

3

The GIS data used in the exclusion zone distance analysis extends 150 kilometers inland and 

120 kilometers along the coast on each side of possible locations for the shipborne radar systems.

The NLCD area classification codes were grouped and mapped to dense urban, urban, suburban, 

and rural regions and applied to 90 meter by 90 meter bins. Next the raw population density data 

for each classification was determined. For the dense urban and urban regions, a daytime 

traveling factor expressed in terms of a percentage was included to account for the higher 

population densities that occur in cities during daytime.

4

To be conservative and assuming a mature deployment phase, a market penetration factor of 

20 percent was assumed for this band. To account for distribution of users across ten available 

10 MHz channels in the 3.5 GHz band, a scaling factor of 10 percent was also included to 

determine the number of effective users potentially operating in each 10 MHz channel of 

interest. 

 3

The population data is based on the 2010 U.S. Census data available at http://www.census.gov/geo/mapsdata/data/gazetteer2010.html and 2011 National Land Cover Database data is available at http://www.mrlc.gov/. 4

The daytime commuter factors available at https:///www.census.gov/hhes/commuting/data/daytimepop.html were 

used as a baseline.
5

The relative percentages of indoor and outdoor APs and UE varies depending on the 

classification regions. For APs and UE in dense urban/urban regions, 80 percent of the APs and 

UE were assumed to be deployed indoors, while 20 percent of the APs and UE were assumed to 

be deployed outdoors. For APs and UE in suburban and rural regions, 99 percent of the APs and 

UE were assumed to be deployed indoors, while 1 percent of the APs and UE were assumed to 

be deployed outdoors. The locations and antenna height of the AP and UE were treated as 

random variables in the analysis.

The antenna height for outdoor APs was fixed at 6 meters. For indoor APs an antenna height 

distribution across different floors was assumed with higher probability of lower floors, as they 

apply to more buildings, and lower probability of higher floors, as they apply to fewer buildings 

in city environments.

Given the limited time and available data, and for simplicity, no region-specific building data 

was used. The same sets of assumptions were equally applied to all regions and cities considered 

in the analysis.

In addition, the following factors were associated with the number of effective users per AP 

corresponding to each region classification:

• For the dense urban/urban classification, 50 effective users per AP

• For the suburban classification, 20 effective users per AP

• For the rural classification, 3 effective users per AP

More users share the coverage of each AP in enterprise and downtown areas. In rural areas, most 

APs serve a single family dwelling with an average of three family members per dwelling.

The total number of APs for each of the classification regions was computed using the following 

equations:

NNNNNNNNNN oooo AAAAAA (DD DD UU UU /UU UU ) = × �1 + DD

100� ×

MM

100 ×

CCCC

100

50

(1)

NNNNNNNNNN oooo AAAAAA ( ) = ×

MM

100 ×

CCCC

100

20

(2)

NNNNNNNNNN oooo AAAAAA ( ) = ×

MM

100 ×

CCCC

100

3

(3)

where

is the raw census population for a region class; 

DD is the daytime commuter adjustment factor (percentage);
6

MM is the market penetration factor (percentage); and

CCCC is the channel scaling factor (percentage).

An example of the AP calculations for San Diego, CA is provided in Table 2, where in this case 

the population in the urban and dense urban regions was augmented by 15 percent to account for 

daytime commuter adjustment. 

Table 2. Example Calculation for Number of APs 

Region Population MP CS D AP/User Number of APs

Urban 4,715,098 0.2 0.1 115 0.02 2,169

Suburban 1,696,558 0.2 0.1 100 0.05 1,697

Rural 653,558 0.2 0.1 100 0.333 4,353

It is understood and expected that AP counts and distribution in each region vary widely based 

on deployment assumptions and service penetration growth over time. Given the critical nature 

of incumbent use in the 3.5 GHz band, and that the transition time to eventually implement 

sensing and dynamic protection zones is not known in advance, a conservative approach was 

used in the assumptions made in this analysis. 
7

3. RADAR TECHNICAL AND DEPLOYMENT PARAMETERS

3.1 Radar Technical Parameters

In the Fast Track Report [1], NTIA performed an analysis to determine the exclusion zone 

distances necessary to protect federal shipborne and ground-based radar systems. The shipborne 

radars analyzed in the Fast Track Report were designated as Shipborne Radars 1 through 5 and 

the ground-based radars were designated GB-1 and GB-3. Shipborne Radars 2 and 3 operate 

below 3550 MHz and Shipborne Radars 1, 4, and 5 are capable of operating in the 3.5 GHz band. 

The ground-based radars both operate below 3550 MHz. The technical parameters for the radar 

receivers used in the exclusion zone distance analysis include: 3 dB intermediate frequency (IF) 

filter bandwidth, noise figure, mainbeam antenna gain, antenna 3 dB beamwidth (vertical and 

horizontal), insertion/cable losses, and antenna height. The 3.5 GHz Study uses these same 

assumptions and names.

In the 3.5 GHz Study analysis, the generalized mathematical model of the radar systems antenna 

described in Recommendation ITU-R M.1851 was used to determine the radar receive antenna 

gain in the azimuth and elevation orientations in the direction of the APs or UE [6]. Knowledge 

of the antenna 3 dB beamwidth and first peak side-lobe level, allowed selection of equations for 

both the azimuth and elevation patterns. Cosine patterns were used for the radar systems’

antennas. The equations for the theoretical antenna directivity parameters of the model are 

provided in Table 3; the equations for the peak and average mask patterns used in the model are 

provided in Table 4.

Table 3. Antenna Directivity Parameters 

Relative shape 

of field 

distribution f(x) 

where −1 ≤ x ≤ 1

Directivity 

pattern F(μ)

θθ33, the 3 dB or half power 

beam-width (degrees)

μ as a 

function of θθ33

First sidelobe level 

below 

main lobe 

peak (dB)

Proposed 

mask floor 

level

(dB)

cos �

ππ

2 xx� ππ

2 � cos μμ

�

ππ

2�

2

− μμ2

�

68.8 �

λλ

ll

�

λλ is the wavelength

ll is the overall length of the 

aperture

68.8ππ

sin θθ

θθ3

-23 -50

Table 4. Peak and Average Mask Pattern Equations

Pattern Type

Mask equation 

beyond pattern 

break point where 

mask departs from 

theoretical pattern

(dB)

Peak pattern break 

point where mask 

departs from 

theoretical pattern

(dB)

Average pattern 

break point where 

mask departs from 

theoretical pattern

(dB)

Constant added to 

the peak pattern to 

convert it to average 

mask

(dB)

Cosine −17.51 ln �2.33 |θθ|

θθ3

� -14.4 -20.6 -4.32
8

The following equation was used to model the off-axis antenna gain pattern for the shipborne 

radar systems:

GG(θθ) = GGMMMMMM − 12 � θθ

BBBB�

2 (4)

where

GG(θθ) is the antenna gain at the off-axis angle of θθ in degrees (dBi);

GGMMMMMM is the mainbeam antenna gain (dBi); and

BBBB is the antenna beamwidth (degrees).

The interference thresholds used to assess compatibility are referred to as long-term thresholds 

because their derivation assumes that the interfering signal levels are present most of the time.

The interference threshold, IITT, used in computing the exclusion distances to protect federal radar 

systems was determined using (5) (see [7] (15)):

IITT = +

II (5)

where

II

NN is the maximum permissible interference-to-noise ratio at the receiver IF output 

necessary to maintain acceptable performance criteria (dB); and

is the receiver noise power level at the receiver IF output in the IF bandwidth (dBm).

For a receiver 3 dB IF bandwidth and receiver noise figure the receiver noise power level is 

given by (6) (see [7] Section 3.2):

= −114 + 10 log BBBBrr + NN (6)

where

BBBBrr is the receiver 3 dB IF bandwidth (MHz); and

NN is the receiver noise figure (dB).

3.2 Radar Deployment Parameters 

In the analysis to compute the exclusion zone distances for the shipborne radar systems, virtual 

ships were positioned at the various locations along the East, West and Gulf coasts. A limited 

number of shipborne radar ports of call and test sites were also considered. Ground-based radars 

were positioned at their normal test and training sites. (In practice, ground-based radars may redeploy periodically to conduct training or for homeland defense missions.)
9

4. DESCRIPTION OF SHIPBORNE RADAR ANALYSIS METHODOLOGY

4.1 Analysis Overview

A Monte Carlo analysis was used to compute exclusion zone distances on each of the individual 

azimuths from the shipborne radars.

5 APs or UE were randomly located in the previously 

described 90 meter by 90 meter bins corresponding to the different regions (dense urban/urban, 

suburban, and rural) and exclusion distances were computed. The Monte Carlo analysis used

10,000 iterations to compute the exclusion zone distance on each azimuth, each time randomly 

varying the different parameters. The parameters randomized in the analysis included: AP and 

UE location, indoor AP antenna heights, building attenuation losses, clutter losses, time 

variability and location variability associated with the propagation loss and the radar receive 

antenna gain. 

The analysis placed ships at coastal locations 10 kilometers offshore. At each ship’s location, the 

azimuth angle of the radar receive antenna mainbeam was varied in 1 degree increments, pointed 

at areas along the coast. This set of inland azimuth angles of the radar antenna mainbeam toward 

the APs and UE remained fixed during each iteration of the Monte Carlo analysis. The aggregate 

interference calculations used to determine the exclusion zone distances were computed on each 

of the individual azimuth angles generated at each ship location.

4.2 Interference Link Budget

The interference power levels at the radar system receiver were calculated using (7) for each AP 

or UE transmitter considered in the analysis:

II = EE EE AAAA/UUUU + GGRR − LLTT − LLRR − LL − LLCC − LLBB − (7)

where

II is the received interference power at the output of the antenna (dBm);

EE EE AAAA/UUUU is the EIRP of the AP or UE transmitter (dBm);

GGRR is the radar receive antenna gain in the direction of the AP or UE transmitter (dBi);

LLTT is the AP transmitter insertion loss (dB);

LLRR is the radar receiver insertion loss (dB);

LL is the basic transmission loss (dB);

LLCC is the clutter loss (dB, only applied to rural sources);

 5 Monte Carlo methods are a class of algorithms that rely on repeated random sampling to compute the solution to 

problems whose solution space is too large to explore systematically or whose systemic behavior is too complex to 

model.
10

LLBB is the building attenuation loss (dB, only applied to indoor sources); and

is the frequency dependent rejection (dB).

Using (7), the values of interference power level were calculated for each AP or UE transmitter 

considered in the analysis. These individual interference power levels were then used in the 

calculation of the aggregate interference to the radar system receiver and converted to dBm using 

(8).

6

IIAAAAAA = 10 log ��IIjj

NN

jj=1

� + 30

(8)

where

IIAAAAAA is the aggregate interference level at the radar receiver from the AP or UE

transmitters (dBm);

is the number of AP or UE transmitters; and

IIjj is the interference power level at the input of the radar receiver from an individual AP 

or UE transmitter (Watts)

The difference between the received aggregate interference power level computed using (8) and 

the radar receiver interference threshold represents the available margin. When the available 

margin is positive, compatible operation is possible. The distance at which the available margin 

is zero represents the minimum distance separation that is necessary for compatible operation.

This distance was used to establish the exclusion zone distances to protect the federal radar 

systems.

4.3 Radar Receiver Interference Threshold

The desensitizing effect on a radar system due to interference from other services with a noiselike modulation such as those from the AP or UE transmitters is predictably related to its 

intensity. In any azimuth in which such interference arrives, its power spectral density can, to 

within a reasonable approximation, simply be added to the power density of the radar receiver 

thermal noise. In the exclusion zone distance analysis, the interference threshold for the 

shipborne radar was based on an II

NN

criterion of -6 dB [4]. An II

NN

of -6 dB corresponds to a 1 dB

 6 The interference power calculated in (7) must be converted from dBm to Watts before calculating the aggregate 

interference seen by the radar receiver using (8). The constant term on the right-hand side of (8) provides the 

conversion back to dBm.
11

increase in the radar receiver noise level.

7 The interference level was based on the aggregate 

interference from AP or UE transmitters.

8

4.4 AP and UE EIRP Values

The EIRP for outdoor APs used in the analysis were assigned based on the dense urban/urban, 

suburban, and rural classifications. Outdoor APs in urban/urban regions were assigned an EIRP 

of 27.8 dBm. Outdoor APs in suburban regions were assigned an EIRP of 26 dBm. Outdoor APs 

in rural classifications were assigned an EIRP of 23 dBm.

Indoor APs were all assigned an EIRP of 26 dBm.

The UE in the analysis were all assigned an EIRP of 24 dBm. 

4.5 Radar Receive Antenna Gain

The antenna gain model described in Recommendation ITU-R M.1851 was used to determine the 

antenna gain in the azimuth and elevation orientations for the radar systems. The radars have the 

ability to track down to the horizon, so a 0 degree up-tilt angle (e.g., 0 degree elevation angle) 

was used in the analysis.

4.6 Transmitter and Receiver Insertion Losses 

In the analysis, an additional factor of 2 dB was included for insertion loss, cable loss, etc., for 

the outdoor APs. An additional 2 dB was used for additional losses associated with the radar 

receivers.

4.7 Propagation and Clutter Loss

In the aggregate interference analysis, it was recognized that interference predictions arising 

from sources in a small cell deployment in heavy traffic areas would be required to account for 

radio propagation in man-made and naturally cluttered environments, as well as other 

impediments to propagation, such as terrain obstructions. An extensive review was performed of 

existing propagation models. In general, it was found that most of the existing propagation 

models were used for predicting signal strength and propagation path loss in built-up 

urban/suburban areas where there are numerous man-made building structures. Typically these 

propagation models are based on measurements with a high antenna (e.g., base station) and a 

lower antenna (e.g., mobile station) immersed in clutter. Propagation models based on this 

methodology (i.e., using the mean/median of measurements at given distances between the two 

terminals) tend to underestimate interference for the small percentages of time/locations, which 

must be considered for interference calculations. It was understood that accurate interference and 

 7 An increase in the receiver noise can reduce the operating range or increase the tracking error of a radar system. 8 NTIA has published several reports that confirm a II

NN of -6 dB is the appropriate protection criterion for radar 

receivers in the presence of noise-like interfering signals [8]–[10]. 
12

propagation models should be developed and tuned based on real field measurement results.

However, given limitations on time and resources and after consideration of possible alternative 

models and the aggressive schedule of the work, a compromise approach was adopted as the way 

forward. This compromise was to revisit the Okumura et al. [11] basic median attenuation 

curves, with the intention of extending Hata’s [12] empirical formulae in both distance and 

frequency ranges, and then apply the “Urban Factor” approach suggested by Longley [13], a 

method referred to as the extended Hata model. A brief description of this approach is provided 

below.

To extend the Hata [12] model in frequency and distance, it was assumed that the base and 

mobile station height gain terms appearing in the empirical formulae were unchanged. Then it is 

sufficient to extend the frequency range of the formulae by refitting to frequencies from 1,500–

3,000 MHz, based on the Okumura et al. [11] median basic attenuation curves at the reference 

base and mobile station heights (see [11] Figure 15). For path distances less than approximately 

20 km, this was accomplished by simply refitting the basic median attenuation curve at 1 km 

distance. However, it is also important to consider path distances longer than approximately 20 

km. For these distances, the approach was to fit the basic median attenuation curve at 100 km 

distance, and then use the long distance exponents shown in [11] Figure 12, Curve B, to 

determine the distance at which the two power laws intersect (i.e., the break-point distance).

Effectively, for distances less than or equal to the break-point distance, the extended Hata model 

uses the original Hata power law exponent and the (refitted) Hata intercept, while for distances 

greater than the break-point distance, the extended Hata model uses the long distance power law 

exponent and the 100 km basic median attenuation curve fit.

In addition to the extensions in the frequency and distance ranges, the following site-specific 

corrections to the median attenuation ([11], Section 4) were implemented:

• AP/UE “effective height” corrections

• “Median” correction for rolling hilly terrain (applied in the vicinity of the AP/UE only)

• “Fine” correction for rolling hilly terrain (applied in the vicinity of the AP/UE only)

• General slope of terrain correction (applied in the vicinity of the AP/UE only)

• Isolated mountain (or isolated ridge) correction (applied only to single horizon paths) 

• Mixed land-sea path correction

These corrections are site-specific in that they depend on the details of the terrain profile 

between the AP/UE. The environment surrounding the AP/UE (urban or suburban) was also 

accounted for via the [11] Figure 20 correction for suburban environments relative to urban 

environments.

For APs less than 18 meters in height above ground in urban and suburban environments, the 

extended Hata model median basic transmission loss was then compared to the ITS Irregular 

Terrain Model (ITM) predicted median basic transmission loss (in point-to-point mode) and the 
13

larger value was used as the basis for Monte Carlo studies of the aggregate interfering power

[14]. For APs in rural environments at all heights and in urban and suburban environments above 

18 meters height, the ITM median basic transmission loss was the basis for the Monte Carlo 

studies. For rural APs, an additional clutter factor of 0–15 dB was randomly applied, with a 

uniform distribution, on each Monte Carlo iteration9. The specific implementation of the 

propagation loss models and clutter loss used in the exclusion zone distance analysis is described 

in Appendix A.

4.8 Building Attenuation Loss

In the dense urban/urban region, it was assumed that 80 percent of the APs and UE will operate 

indoors. In the suburban and rural regions, it is assumed that 99 percent of the APs and UE will 

operate indoors. For the indoor APs, a building attenuation loss in addition to the propagation 

loss was used in the exclusion zone distance analysis. The following random distribution was 

used to represent the building attenuation loss in the analysis: 20 percent were assigned a 

building attenuation loss of 20 dB; 60 percent were assigned a building attenuation loss of 15 dB; 

and 20 percent were assigned a building attenuation loss of 10 dB.

4.9 Frequency Dependent Rejection

Frequency dependent rejection (FDR) accounts for the fact that not all of the undesired 

transmitter energy at the receiver input will reach the detector. FDR is a calculation of the 

amount of undesired transmitter energy that is rejected by a victim receiver. This FDR 

attenuation is composed of two parts: on-tune rejection (OTR) and off-frequency rejection 

(OFR). The OTR is the rejection provided by a receiver selectivity characteristic to a co-tuned 

transmitter as a result of an emission spectrum exceeding the receiver bandwidth. A detailed 

description of how to compute FDR can be found in [15]. The transmitter emission spectrum and 

receiver selectivity curves used in the FDR calculation are defined in terms of a relative 

attenuation level specified in decibels as a function of frequency offset from center frequency in 

megahertz.

The FDR can be stated mathematically in decibel terms as:

= 10 log � ∫ pp( − tt )dd ∞

0

∫ pp( − tt )h( − rr )dd ∞

0

�

(9)

where

tt is the undesired transmitter tuned frequency;

rr is the receiver tuned frequency;

pp( ) is the normalized emission spectrum of the undesired transmitter;

 9

The clutter factor is being applied to account for additional losses (e.g., due to vegetation) for low APs in rural 

environments.
14

h( )is the normalized transfer function of the receiver; and

is the absolute frequency.

Numerical integration and convolution routines were used to solve (9).

In the special case of an undesired transmitter operating co-channel to a receiver, the following 

simplified form may be used.

= �0,10 log �

BBtt

BBrr

�� (10)

where

BBtt is the emission bandwidth of the undesired transmitter (MHz); and

BBrr is the IF bandwidth of the receiver (MHz).

For Shipborne Radars 1, 4, and 5, all of the APs or UE considered in the exclusion zone distance 

analysis were operating co-frequency with the radar systems and (10) was used to compute the 

FDR. For Shipborne Radars 2 and 3 an approximate frequency separation of 50 MHz from the 

APs or UE and (9) was used to compute the FDR.

4.10 Shipborne Radar Exclusion Zone Distance Analysis Output

The Monte Carlo analysis produced a distribution of results. Figure 1 shows an example plot for 

a single ship location, comparing the results for various confidence intervals. The minimum 

(blue curve), maximum (red curve), mean (yellow curve), and 95 percent (green curve) exclusion 

zone distances on each of the individual azimuths are shown on Figure 1. The maximum, 

minimum, and percentile values are derived for the 10,000 iterations for each azimuth. The 

minimum value is the smallest exclusion zone distance for the 10,000 iterations. The maximum 

is the largest exclusion zone distance for the 10,000 iterations at each azimuth. The 95 percent 

value is the exclusion zone distance that is exceeded by 5 percent of the iterations at that 

azimuth. Figure 2 is an example of the exclusion zone distance analysis output.
15

Figure 1. Example—Exclusion zone distances referenced to the coastline for different confidence 

intervals.

0

20

40

60

80

100

120

140

305313321329337345353

19

1725334149576573818997

105113121129

Exclusion Zone from Shore (km)

Azimuth (Degrees)

San Diego - Shipborne Radar 1 - 10,000 Iterations

Max

Mean

Min

Pct 95
16

Figure 2. Example—Exclusion zone distances for different confidence intervals for a shipborne 

radar at S2.

The 95 percent distance calculations were used to establish the exclusion zones to protect the 

shipborne radar. The confidence interval was used to quantify the uncertainty in a Monte Carlo 

statistical analysis. The confidence interval can be computed by [16]: 

CC CC CC II II II = pp ± zz�pp(1 − pp) (11)

where

pp is the estimated probability (0.95); 

is the number of samples (i.e., iterations in the Monte Carlo analysis (10,000)); and 

zz is the constant multiplier depending on the chosen confidence interval (1.96 for a 

normal distribution).10

 10 The assumption of a normal distribution is reasonable as long as > 5 and (1 − pp) > 5.
17

Using these factors, the confidence interval for this analysis is 95 percent ± 0.43 percent. The 

expected 95 percent value for the exclusion zone distance is between 94.6 percent and 95.4 

percent for the 10,000 iterations with a confidence level of 95 percent.

It is understood that such a high percentage is not common for interference analysis in designing 

commercial systems. But, given the critical missions of federal incumbent systems operating in 

the 3550–3650 MHz band, it was agreed following some deliberations to take a conservative 

approach and use a higher percentage for establishing the exclusion zone distances. The results 

of the shipborne radar exclusion zone distance analysis are presented in Appendix B.
18

5. DESCRIPTION OF GROUND-BASED RADAR ANALYSIS METHODOLOGY

The analysis used to compute the exclusion zone distances for ground-based radars employed the 

Monte Carlo analysis capabilities of Visualyse (©Transfinite Systems Ltd.), a commercial 

modeling and simulation application. A less detailed analysis approach was taken for the ground 

based radars, since they operate below 3550 MHz and there will be additional interference 

rejection due to the filters used in the ground based radar receiver and the CBRS AP transmitter. 

The analysis only considered outdoor APs with the ground-based radar placed at the edge of a 

city near the base on which the radar could be deployed. The APs were randomly distributed 

within one or more square regions covering a portion of the heavily populated city area at which 

the radar antenna mainbeam was pointed, as illustrated in Figure 3. The number of APs in this 

area was calculated from the 2010 U.S. census population data using (12). 

NNNNNNNNNN oooo AAAAAA = × MM × × OO × CCCC

20

(12)

where

is the raw census population for a city; 

MM is the market penetration factor (fraction); 

is the ratio of population in simulation area to total city population (fraction);

OO is the ratio of outdoor APs to total APs (fraction); and

CCCC is the channel scaling factor (fraction).

A market penetration factor of 40 percent, a 20 percent ratio of outdoor to indoor APs, a channel 

usage factor of 100 percent, and a factor of 20 users per AP were used to compute the number of 

APs considered in the analysis. A 100 percent channel usage factor was chosen since the CBSD 

out-of-band emission spectrum used was relatively constant over the allocated band and thus 

CBSDs operating on all channels contributed to the interference level at the radar receiver. The 

percentage of city population included in the simulation area was taken as the ratio of the 

simulation area that included the radar main beam and its near-in sidelobes to the area of the city 

that was heavily developed. The number of APs calculated in (12) was divided equally among 

multiple square areas, as depicted in Figure 3. Visualyse randomly distributed the AP locations 

within each square for each Monte Carlo analysis iteration. The analysis considered 1,000 

random configurations for each exclusion zone distance. The randomized parameters in the 

analysis included: AP location within the specified simulation area, clutter loss (0 to 15 dB), and 

the reliability variability (0 to 100%) used in the propagation loss calculation. A uniform 

probability distribution was used to select the values for these parameters.
19

Figure 3. Example of Visualyse simulation area.

5.1 Interference Link Budget

The interference power levels at the radar receiver were calculated using (13) for each AP 

transmitter considered in the analysis.

II = EE EE AAAA + GGRR − LL − LLCC − LLRR − (13)

where

II is the received interference power at the output of the antenna (dBm);

EE EE AAAA is the EIRP of the AP transmitter (dBm);

GGRR is the radar receive antenna gain in the direction of the AP transmitter (dBi);

LL is the basic transmission loss (dB);

LLCC is the clutter loss (dB);

LLRR is the radar line loss (dB); and 

is the frequency dependent rejection (dB).

The aggregate interference was computed using (8). 

Assumed GB-1 

Radar at

39.1929, -96.6771

Azimuth = 90°

4 km by 8 km Visualyse 

Simulation Area

Simulation Area Center

39.1929, -96.6181

Simulation Area Center

39.1929, -96.5712
20

5.2 AP EIRP and Antenna Height

The EIRP used in the analysis for all of the APs was 30 dBm. An antenna height of 6 meters was 

assumed for all of the APs in the analysis.

5.3 Radar Receive Antenna Gain

The antenna gain model described in Recommendation ITU-R M.1851 was used to determine the 

antenna gain. A receive line loss of 2 dB was assumed. The radar receive antenna gain will vary 

depending on the location of the AP (distance from the radar) and antenna height of the AP. An 

antenna height of 5 meters was assumed for the center-line of the radar antenna. The groundbased radars have the ability to track down to the horizon, so a 0 degree up-tilt angle (e.g., 0 

degree elevation angle) was assumed.

5.4 Basic Transmission Loss

The Longley-Rice terrain dependent propagation model was used to compute the basic 

transmission loss for the exclusion distance analysis. The reliability variability was treated as a 

random variable in the analysis, with a minimum value of 0 percent and a maximum value of 100 

percent selected using a uniform probability distribution.

5.5 Clutter Loss

The clutter loss was treated as a random variable in the analysis with a minimum value of 0 dB 

and a maximum value of 15 dB, selected using a uniform probability distribution. 

5.6 FDR

Interference calculations were performed assuming the radar was operating at 3490 MHz.

Figure 4 (which is based on [5] Figure 74) displays the spectrum emission mask assumed for the 

AP in calculating the FDR. Radar receiver selectivity data was obtained from radar equipment 

certification records.
21

Figure 4. AP emission spectra measurement used for the model and the analysis.

Using this emission spectrum for APs and the receiver selectivity of the ground-based radar, an 

FDR of 67.2 dB was computed for radar GB-1 and 59.3 dB for radar GB-3 and used in the 

analysis. This FDR was used for all the APs considered in the analysis.

5.7 Radar Receiver Interference Threshold

An II

NN

of -6 dB was used as the interference criteria to compute the ground-based radar receiver 

threshold.

5.8 Ground-Based Radar Exclusion Zone Distance Analysis Output

The Monte Carlo analysis was used to compute plots of the cumulative distribution of the 

probability of not exceeding the II

NN

criteria of -6 dB for different exclusion zone distances.

Figure 5 is an example of the exclusion zone distance analysis output.

-70.00

-60.00

-50.00

-40.00

-30.00

-20.00

-10.00

0.00

10.00

-20.00 -10.00 0.00 10.00 20.00

Response in dB

Frequency in MHz

LTE Base Station Spectra (normalized)

Noise Floor of

measurement system
22

Figure 5. Example output–II

NN cumulative distribution of interference probability

The results of the exclusion zone distance analysis for the ground-based radars are presented in 

Appendix C. Based on the analysis, a 3 kilometer exclusion zone around the perimeter of each of 

the installations is needed to protect the ground-based radars.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-25 -20 -15 -10 -5 0 5 10 15

Probability of Exceeding I/N

Oceanside/Carlsbad into Camp Pendleton Aggregate I/N (dB)

Exclusion Zone = 0 km Exclusion Zone = 1 km Exclusion Zone = 2 km

Exclusion Zone = 3 km Exclusion Zone = 4 km

95th percentile -6 dB aggregate I/N
23

6. SUMMARY OF EXCLUSION ZONE DISTANCE ANALYSIS

Based on the analysis results, it was determined that the exclusion zone distances to protect 

shipborne radars from APs would also provide adequate protection from UE. The analysis results 

presented for the shipborne and ground-based radars only consider interference from the APs.

The results for the shipborne radar exclusion zone distance analysis compared to the Fast Track 

Report exclusion zone distances are provided in Tables 5–7. The results are presented for three 

parameters: percentage of area impacted, percentage of population impacted, and average 

exclusion zone distance. Exclusion zones were not necessary to protect Shipborne Radar 2 and 

Shipborne Radar 3 analyzed in the Fast Track Report.

The 3.5 GHz Study’s exclusion zone distance analysis for the ground-based radar analysis is 

shown in Table 8. The exclusion zones for ground-based radars in the Fast Track Report were 

between 40 and 60 km.11

Table 5. Summary of Shipborne Radar Analysis Results

Geographic Area

Percentage of Area Impacted (km2)

Fast Track Report 

Analysis12

Revised Analysis

(3.5 GHz Study)

Percent Reduction of 

Area Impacted

East Coast 1,081,389 345,067 68

West Coast 956,435 118,035 88

Gulf Coast 880,096 217,902 75

Table 6. Summary of Shipborne Radar Analysis Results

Geographic 

Area

Fast Track Report Analysis Revised Analysis Reduction in 

Percentage of 

Total Population 

Impacted

Population 

Impacted

Percentage of 

Total Population 

Impacted

Population

Impacted

Percentage of 

Total Population 

Impacted

East Coast 66,771,384 32 50,566,713 24 24

West Coast 45,145,947 22 32,404,740 16 28

Gulf Coast 29,342,307 14 13,360,186 6 54

Table 7. Summary of Shipborne Radar Analysis Results

Geographic Area

Average Exclusion Zone Distance (km)

Fast Track Report 

Analysis

Revised Analysis

(3.5 GHz Study)

Reduction in Average 

Exclusion Zone Distance

East Coast 427 135 292

West Coast 423 60 363

Gulf Coast 427 68 359

 11 [1] Table 5-2. 12 [1] Table 5-4.
24

Table 8. Summary of Ground-Based Radar Analysis Results

Military Base Radar Nearby City Exclusion Zone

Fort Carson CO GB-1 Colorado Springs CO 2 km

Fort Riley KS GB-1 Junction City KS 3 km

Fort Riley KS GB-1 Manhattan KS 2 km

Fort Sill OK GB-3 Lawton OK 2 km

Camp Pendleton CA GB-3 Oceanside/Carlsbad CA 4 km
25

7. REFERENCES

[1] U.S. Department of Commerce, National Telecommunications and Information 

Administration, An Assessment of the Near-Term Viability of Accommodating Wireless 

Broadband Systems in the 1675-1710 MHz, 1755-1780 MHz, 3500-3650 MHz, and 4200-

4220 MHz,4380-4400 MHz Bands (Nov. 15, 2010) (Fast Track Report), available at 

http://www.ntia.doc.gov/reports/2010/FastTrackEvaluation_11152010.pdf.

[2] Memorandum for the Heads of Executive Departments and Agencies, “Unleashing the 

Wireless Broadband Revolution” (rel. Jun. 28, 2010), published at 75 Fed. Reg. 38387 (Jul. 

1, 2010), available at http://www.whitehouse.gov/the-press-office/presidentialmemorandum-unleashing-wireless-broadband-revolution. 

[3] “Commission Seeks Comments on Shared Commercial Operations in the 3550–3650 MHz 

Band,” Further Notice of Proposed Rulemaking in GN Docket No. 12-354, FCC 14-49, 

Fed. Reg. Vol. 79, No. 105, 31247 (Jun. 2, 2014).

[4] “Amendment of the Commission’s Rules with Regard to Commercial Operations in the 

3550-3650 MHz Band,” Report and Order and Second Further Notice of Proposed 

Rulemaking in GN Docket No. 12-354 (Apr. 21, 2015).

[5] Geoffrey Sanders, John E. Carroll, Frank H. Sanders, and Robert L. Sole, Effects of Radar 

Interference on LTE (FDD) eNodeB and UE Receiver Performance in the 3.5 GHz Band, 

National Telecommunications and Information Administration Technical Report TR-14-

506 (Jul. 2014), available at http://www.its.bldrdoc.gov/publications/2759.aspx. 

[6] International Telecommunication Union-Radiocommunication Sector, Recommendation 

ITU-R M.1851, Mathematical Models for Radio determination Radar Systems Antenna 

Patterns for use in Interference Analyses (Jun. 2009).

[7] International Telecommunication Union-Radiocommunication Sector, Recommendation 

ITU-R M.1461-1, Procedures for Determining the Potential for Interference Between 

Radars Operating in the Radio determination Service and Systems in Other Services (2000-

2003), Annex 1.

[8] Frank H. Sanders, Robert L. Sole, Brent L. Bedford, David Franc, and Timothy Pawlowitz, 

Effects of RF Interference on Radar Receivers, National Telecommunications and 

Information Administration Technical Report 06-444 (Feb. 2006), available at

http://www.its.bldrdoc.gov/publications/06-444.aspx. 

[9] John E. Carroll, Geoffrey Sanders, Frank H. Sanders, and Robert L. Sole, Case Study: 

Investigation of Interference into 5 GHz Weather Radars from Unlicensed National 

Information Infrastructure Devices, Part II, National Telecommunications and Information 

Administration Technical Report 11-479 (Jul. 2011), available at 

http://www.its.bldrdoc.gov/publications/2554.aspx. 
26

[10] Frank H. Sanders, John E. Carroll, Geoffrey Sanders, Robert L. Sole, Robert J. Achatz, and 

Lawrence S. Cohen, EMC Measurements for Spectrum Sharing Between LTE Signals and 

Radar Receivers, National Telecommunications and Information Administration Technical 

Report 14-507 (Jul. 2014), available at http://www.its.bldrdoc.gov/publications/2760.aspx. 

[11] Y. Okumura, E. Ohmori, T. Kawano, and K. Fukuda, “Field strength and its variability in 

VHF and UHF land-mobile radio service,” Rev. Elec. Commun. Lab., 16, 9-10, pp. 825-

873, (Sept.-Oct. 1968).

[12] M. Hata, “Empirical formula for propagation loss in land mobile radio services,” IEEE 

Transactions on Vehicular Technology, VT-29, 3, pp. 317-325 (Aug. 1980).

[13] Anita G. Longley, Radio Propagation in Urban Areas, United States Department of 

Commerce, Office of Telecommunications, OT Report 78-144 (Apr.1978), available at 

http://www.its.bldrdoc.gov/publications/2674.aspx. 

[14] George A. Hufford, Anita G. Longley, William A. Kissick, A Guide to the Use of the ITS 

Irregular Terrain Model in the Area Prediction Mode, National Telecommunications and 

Information Administration Technical Report 82-100, (Apr. 1982), available at 

http://www.its.bldrdoc.gov/publications/2091.aspx. 

[15] International Telecommunication Union-Radiocommunication Sector, Recommendation 

ITU-R SM.337-4, Frequency and Distance Separations (1997), Annex 1.

[16] Allan Agresti and Brent A. Coull, “Approximate is Better than “Exact” for Interval 

Estimation of Binomial Proportions,” The American Statistician, Vol. 52, No. 2, pp. 119-

126 (May 1998), available at http://www.stat.ufl.edu/~aa/articles/agresti_coull_1998.pdf.
27

APPENDIX A DESCRIPTION OF PROPAGATION AND CLUTTER LOSS MODEL

A.1 Introduction

This appendix describes the implementation of the propagation loss and clutter loss models used 

in the exclusion zone distance analysis.

A.2 Propagation Model Overview

Two propagation models were used to predict the individual links’ basic transmission losses used 

in the exclusion zone distance analyses: the extended Hata model and the ITS Irregular Terrain 

Model (ITM) [A-1]–[A-3] The details of the specific implementations of these propagation 

models are described below. For either of these models, the site-specific or point-to-point mode 

of implementation was employed. Therefore, the great circle terrain elevation profile between the 

individual source’s (access point (AP) or user equipment (UE)) location and the shipborne 

radar’s location (usually 10 km out to sea) was extracted and used by the two models.

For the purposes of the aggregate interference power computation (see (8)), either model’s basic 

transmission loss was treated as a random variable, which varied according to the statistics (i.e.,

location variability in the case of the extended Hata model and location and time variability in 

the case of ITM) described below. This randomness in the basic transmission losses was over 

and above other random aspects of the small cell device deployment characteristics, such as 

terminal heights, indoor vs. outdoor siting and, hence, the application of a random building entry 

loss, and whether a device at a given location was actively transmitting.

For APs and UE located in dense urban/urban and suburban environments (see the description of 

small cell device deployment characteristics in Section 2.2), the median basic transmission losses 

calculated using the extended Hata model and ITM were compared and the model yielding the 

greater median basic transmission loss was then used in the computation of the aggregate 

interference power statistics. For dense urban/urban and suburban APs and UE having terminal 

heights greater than 18 m, and all rural APs and UE, only the basic transmission loss from ITM 

was used in the computation of the aggregate interference power statistics. For rural APs and UE 

only, an additional, uniformly distributed, random “vegetative clutter” loss of 0 to 15 dB was 

added to the ITM basic transmission loss.

On any given Monte Carlo iteration, the active AP/UE locations, siting, and terminal heights 

were randomly assigned, as well as the corresponding basic transmission losses. There then 

commenced a sweep over all relevant (i.e., inland) radar antenna main beam azimuths. For each 

individual AP/UE-to-radar link, the (random) basic transmission loss was adjusted by the 

antenna gain of the radar in that direction and any other relevant losses (see (7)), and the 

individual link’s interference powers were power summed as shown in (8), to yield the aggregate 

interference power for that radar antenna main beam azimuth.

To obtain the corresponding exclusion zone distance for that radar antenna main beam azimuth, 

AP/UE transmitters were then removed (i.e., excluded from the power sum calculation of the 
28

aggregate interference) at successively greater distances from the ship’s location, until the 

aggregate interference power fell below the interference-to-noise ratio of -6 dB.

A.3 The Extended Hata Model

Okumura et al. [A-1] conducted a series of land-mobile measurements made over varied, but 

principally quasi-smooth, terrain in urban, suburban and open/rural environments, and then used 

these measurements to derive curves and adjustment methods suitable for land mobile 

propagation predictions over the distance range 1–100 km, the frequency range 100–3,000 MHz, 

base station effective height range 20–1,000 m and mobile station effective height range 1–10 m. 

Hata [A-2] derived empirical formulae that closely approximated the curves-based methods, for 

quasi-smooth terrain only, for a distance range 1–20 km, a frequency range 150–1,500 MHz, a 

base station effective height range 30–200 m and a mobile station effective height range 1–10 m. 

In the interim, Longley [A-3] made use of the Basic Median Attenuations at base/mobile station 

effective antenna heights’ reference values in [A-1] to develop an “Urban Factor” correction for 

the ITS Irregular Terrain Model (ITM) for quasi-smooth terrain. Longley noted that ITM and 

Okumura et al. yielded similar results with respect to base station effective heights [A-3]. 

The Hata [A-2] empirical formula for the median basic transmission loss for the mobile station 

located in a large city, LLpp, in decibels is given as follows ( ≥ 400 MHz):

LLpp = 69.55 + 26.16 log − 13.82 log hbb − (hmm) + (44.9 − 6.55 log hbb) log dd (A-1)

where

(hmm) = 3.2(log(11.75hmm))2 − 4.97 (A-2a)

while for a mobile station in a medium-small city

(hmm) = (1.1 log − 0.7)hmm − 1.56 log + 0.8 (A-2b)

and the units are: [ ]=MHz, [dd]=km and [hbb, hmm]=m. For a mobile station located in a suburban 

area, [A-2] provides a fit to the [A-1] suburban correction factor:

LLpp,ss = LLpp − 2 �log �

28��

2

− 5.4

(A-3)

As noted in [A-2], (A-1) is a slope-intercept form for LLppwith respect to the independent variable 

log dd, so the extension of the frequency range may be viewed as a matter of adjusting the first 

two terms on the right-hand side of (A-1) to match the intercept’s behavior at higher frequencies, 

if the slope and height gains are unchanged. A quick inspection of Figure 15 in [A-1] suggests a 

new curve fit of the form: αα + ββ log + γγ(log )2. The coefficients of the fit can be found using 

the simultaneous solution of an equal number of equations and unknowns, given the values of the 

intercepts at three independent frequencies: 1,500 MHz, 2,000 MHz and 3,000 MHz.
29

Note that the values of the basic median attenuation relative to free space shown in Figure 15 of 

[A-1] are given at the reference antenna effective base and mobile stations’ heights of 200 m and 

3 m, i.e., hbb and hmm, respectively. The basic median attenuations at 1 km are 22 dB, 23.5 dB and 

25.85 dB for the three frequencies: 1,500 MHz, 2,000 MHz and 3,000 MHz, respectively. Then 

the three equations in the three unknown coefficients are:

22 = αα1 + ββ1 log(1500) + γγ1(log(1500))2

− 13.82 log(200) − 3.2(log(35.25))2 + 4.97

− LLff (1500, )

(A-4a)

23.5 = αα1 + ββ1 log(2000) + γγ1(log(2000))2

− 13.82 log(200) − 3.2(log(35.25))2 + 4.97

− LLff (2000, )

(A-4b)

25.85 = αα1 + ββ1 log(3000) + γγ1(log(3000))2

− 13.82 log(200) − 3.2(log(35.25))2 + 4.97

− LLff (3000, )

(A-4c)

where

LLff ( , ) ≅ 20 log �4ππ 300� = 20 log �

4ππ

300� + 20 log (A-4d)

with

= �(1 × 103)2 + (hbb − hmm)2 = �106 + (200 − 3)2 (A-4e)

The unknown coefficients are:

αα1= 97.62;

ββ1 = 3.19;

γγ1 = 4.45.

The resulting extended Hata median basic transmission loss for a mobile station in a large city 

urban area is thus 1,500 MHz ≤ ≤ 3,000 MHz and 1 km ≤ dd ≲ 20 km:

LLpp

EE = 97.62 + 3.19 log + 4.45(log )2 − 13.82 log hbb

− 3.2(log(11.75hmm))2 + 4.97 + (44.9 − 6.55 log hbb) log dd

(A-5)

The corresponding basic median attenuation relative to free space, AAbb

EE (1,500 MHz ≤ ≤

3,000 MHz and 1 km ≤ dd ≲ 20 km), is given by:

AAbb

EE( , dd) ≅ 30.52 − 16.81 log + 4.45(log )2 + 9.83 log dd (A-6)
30

Turning next to the extension of the Hata model for distances greater than 20 km, it is more or 

less clear why Hata [A-2] limited his empirical formulas to the distance range 1–20 km 

originally. A careful review of [A-1] reveals that the measurements showed different distance 

power law exponents for distances greater than ~20–40 km (e.g., see Figures 11, 12 and 15 of 

[A-1]). The dependence of the median attenuation’s distance power law exponent on the base 

station effective height for distances above about 20 km, h(hbb),, is also fundamentally different 

from its behavior with base station effective antenna height for distances below 20 km, ll(hbb).

The distance extension begins with a curve fit of the basic median attenuation relative to free 

space at 100 km, AA( , 100) = 10 log ( , 100), takes the functional form: αα + ββ log +

γγ(log )2, using values taken from Figure 15 of [A-1] at 1,500 MHz, 2,000 MHz and 3,000 

MHz. These are 63.5 dB, 65.75 dB and 69.5 dB, respectively. The resulting values for the curve 

fit coefficients are:

αα100 = 120.78;

ββ100 = -52.71;

γγ100= 10.92.

From (A-1) and (A-5), the base station effective height dependence of the lower distance range 

power law exponent of the median attenuation relative to free space, ll(hbb), is ll(hbb) =

0.1(24.9 − 6.55 log hbb). To obtain the higher distance range power law exponent of the median 

attenuation relative to free space, h, use Curve B in Figure 12 of [A-1]. The required curve fit 

of this exponent to a functional form is written as:

h

2 + 1 = ρρ + σσ log hbb + ττ(log hbb)2 (A-7)

The curve fit uses the following three � hbb,

nnh

2 � ordered pairs: (24.5, 2.5), (70.0, 3) and (200, 

3.22). The resulting values for the curve fit coefficients are:

ρρ= -0.75;

σσ = 3.27;

ττ = -0.67.

For this study, the benefits of constructing a smoothly varying distance power law exponent were 

outweighed by the difficulties inherent in such a construction. Instead, a “two-slope” solution 

was chosen: the slope would be determined by the original Hata values below 20 km, and a 

second slope based on the power law exponents in Figure 12, Curve B, of [A-1] are applied to 

the (frequency extrapolated) basic median attenuation relative to free space, bb , at the reference 

base and mobile stations effective heights to find a distance “break-point”, assuming that the 

other height gain functions were unchanged. In this scheme the attenuations (in physical terms) 

can be rewritten as:
31

bb ,ll( , dd) = bb ,ll� , ddll

rrrr � � dd

ddll

rrrr �

0.1(24.9−6.55 loghbb)

= bb ,ll� , ddll

rrrr �� dd

ddll

rrrr �

nnll(hbb)

(A-8a)

for 1 ≤ dd ≲ 20 kk , and

bb ,h( , dd) = bb ,h� , ddh

rrrr � � dd

ddh

rrrr �

nnh(hbb) (A-8b)

for 20 ≲ dd ≤ 100 km.

For convenience, set ddll

rrrr =1 km and ddhrrrr = 100 km. To find the “break-point” distance, 

equate (A-8a) and (A-8b) (and suppress all but the essential functional dependences):

bb ( , 1)ddbb

nnll = 10−2nnh bb ( , 100)ddbbnnh (A-9a)

or, after a little manipulation

ddbb = �102nnh bb ( , 1)

bb ( , 100)

�

1

(nnh−nnll) (A-9b)

Given the “break-point” distance, ddbb , rewrite the extended Hata median basic transmission loss, 

LLpp

EE (1,500 MMMM ≤ ≤ 3,000 MMMM , 1 kk ≤ dd ≤ 100 kk , 30 ≤ hbb ≤ 200 , 1 ≤ hmm ≤

10 ), as:

LLpp

EE( , dd, hbb, hmm)

= AAbb

EE � , ddbb � + 10 log � dd

ddbb

� + 13.82 log �

200

hbb

� + (3)

− (hmm) + LLff � , (dd, hbb, hmm)�

(A-10)

AAbb

EE � , ddbb � = 30.52 − 16.81 log + 4.45(log )2

+ (24.9 − 6.55 log hbb) log ddbb

(A-11)

(dd, hbb, hmm) = �(dd × 103)2 + (hbb − hmm)2 (A-12)

= � 0.1(24.9 − 6.55 log hbb) for 1 kk ≤ dd ≤ ddbb

2(3.27 log hbb − 0.67(log hbb)2 − 1.75) for ddbb < dd ≤ 100 kk

(A-13)

and it is understood that (A-2a) is used for the mobile station’s reference height correction, (3).
32

The lower frequency limit on the extended Hata formula in (A-10) is not absolute but only 

supplied in acknowledgement of the excellent agreement between the Hata formulae and the 

Okumura et al. results from 150 MHz to 1,500 MHz. If it is tolerable to allow small differences 

between these results and (A-1), then it is easy to show that the difference between the two is 

within 1.6 dB over virtually all of the original Hata model’s frequency range.

The suburban factor shown in Figure 20 of [A-1] can be treated in a similar fashion to the 

method used to extend the frequency range of the formula in [A-2]. Fitting the suburban factor to 

a functional form of αα + ββ log + γγ(log )2 at the frequencies 1,500 MHz, 2,000 MHz, and 

3,000 MHz yields the coefficients:

ααss = 54.19;

ββss = -33.30;

γγss = 6.25.

The corresponding extended Hata median basic transmission loss for a mobile station in a 

suburban area is thus 1,500 MHz ≤ ≤ 3,000 MHz and 1 km ≤ dd ≤ 100 km:

LLpp,ss

EE = LLppEE − (54.19 − 33.30 log + 6.25(log )2) (A-14)

which can be seen to alter the frequency dependence of the intercept.

The extended Hata results given above are intended for use in predicting the median basic 

transmission loss in quasi-smooth terrain. Okumura et al. [A-1] also introduced various 

corrections to the median field strength curves, many of which were intended for use in 

situations where the terrain becomes irregular and rugged. These corrections provide a set of 

site-specific adjustments to be applied when terrain data is available for the path in question. 

These corrections are:

• The terminals’ “effective height” corrections

• The “median” correction for rolling hilly terrain (applied in the vicinity of the mobile station 

only)

• The “fine” correction for rolling hilly terrain (applied in the vicinity of the mobile station 

only)

• The general slope of terrain correction (applied in the vicinity of the mobile station only)

• The isolated mountain (or isolated ridge) correction (applied only to single horizon paths)

• The mixed land-sea path correction

The terminals’ “effective height” corrections are defined as the difference between the height of 

the terminal above mean sea level (the sum of the terminal’s structural height above ground and 

the height of the terrain below the terminal above mean sea level) and the average height of 
33

terrain above mean sea level from 3 to 15 km distance from the terminal in question. These 

quantities are to be used in the height gain terms in the extended Hata formulas. For paths less 

than 3 km, the terminal’s structural height is used. For path distances between 3 and 15 km 

length, the average ground height is weighted by fraction of the path distance relative to 15 km.

For both the “median” and “fine” rolling hilly terrain corrections, it is necessary to compute the 

terrain irregularity parameter, ∆h, for the terrain within 10 km of the mobile station. Note that 

Okumura et al. [A-1] refer to ∆h as the terrain undulation parameter. The terrain irregularity 

parameter is defined as the interdecile range (i.e., the terrain height which is exceeded for 10% of 

the terrain elevations minus the terrain height which is exceeded for 90% of the terrain 

elevations) of terrain heights above mean sea level. If the path is less than 10 km in distance, 

then the asymptotic value for the terrain irregularity is computed:

∆h = ∆h(dd) (1 − 0.8 −0.2)

(1 − 0.8 −0.02dd)

(A-15)

For the “median” rolling hilly terrain correction, the median basic transmission loss is increased 

(and presented graphically in [A-1] Figure 28(c)) by:

KKh = 1.507213 − 8.458676 log ∆h + 6.102538(log ∆h)2 (A-16)

For the “fine” rolling hilly terrain correction, it is necessary to compare the mobile station’s 

terrain height to the value of median terrain height for the terrain within 10 km of the mobile 

station (i.e., the terrain height which is exceeded for 50% of the terrain elevations). The “fine” 

rolling hilly terrain correction is negative/positive in terms of its contribution to the median basic 

transmission loss depending on whether the mobile station’s terrain height is greater/less-than the 

median terrain height exceedance level. The maximum magnitude of the correction is given by 

(and presented graphically in [A-1] Figure 29):

KKhff = −11.728795 + 15.544272 log ∆h − 1.8154766(log ∆h)2 (A-17)

which is achieved for mobile station terrain heights equal to the terrain heights exceeded for 10% 

or 90% of terrain elevations, respectively, passing through zero at the median terrain height, with 

linear interpolations between these values. For path distances less than 10 km, the 10%, 50%,

and 90% terrain height exceedance levels follow the asymptotic form used for the terrain 

irregularity parameter in (A-15).

For the general slope of terrain correction, [A-1] found that this correction depends on the 

average slope of the terrain for at least 5 km and up to 10 km on the mobile station’s end of the 

path and the overall path distance (see [A-1] Figure 34). In this correction, terrain which rises, on 

average, in the direction from the base station to the mobile station reduces the median basic 

transmission loss for the flat, or zero slope, case, while terrain which falls, on average, in the 

direction from the base station to the mobile station increases the median basic transmission loss 

for the zero slope case. However, unlike the “fine” rolling hilly terrain correction, the general 

slope of terrain correction is not perfectly antisymmetric. In the current implementation, a least 

squares fit was performed on the terrain elevations in intervals of 5, 6, 7, 8, 9, and 10 km on the 

mobile station’s end of the path. There are then three possible outcomes: all six slopes are 
34

positive, all six slopes are negative, or the slopes have a mixture of signs. If all six are positive, 

then we use the maximum slope; if all six are negative, then we use the minimum slope; if the six 

have a mixture of signs, then we use the slope found for the 5 km portion of the path. Note, if the 

path distance is less than 5 km, then no general slope of terrain correction is applied. Piecewise 

linear curve fits of the correction curves shown in Figure 34 of [A-1] are used, with linear 

interpolation for distances between the curves.

For the isolated ridge correction, [A-1] found that the corrections given above were inadequate if 

the path contained a single isolated ridge. For those paths, they provided a family of three 

correction curves for a ridge height normalized to 200 m, based on the distance of the mobile 

station to the peak of the isolated ridge. The three curves corresponding to distances of the base 

station to the peak of the isolated ridge: ≤15 km, 30 km and ≥60 km (see [A-1] Figure 31). In the 

current implementation this correction applies for single horizon paths only and piecewise linear 

fits are used based on discrete values taken from the curves. To correct for the actual ridge height 

([A-1] Figure 32) the height of the ridge above the line between the electrical centers of the base 

and mobile stations was used. For distances between 15 km and 30 km and 30 km and 60 km, 

interpolate between the applicable curves. 

For the mixed land-sea path correction, Okumura et al. (see [A-1] Figure 35) observed that there 

was a dependence on the fraction of the path over sea, the overall path distance (less than 30 km 

and greater than 60 km) and which terminal, either the base station or the mobile station, was 

principally adjacent to the sea. For intermediate cases, Okumura et al. suggested that linear 

interpolation gives satisfactory results. The current implementation uses piecewise linear fits to 

the curves in [A-1] Figure 35. In all cases where the fraction of the path over sea is non-zero, the 

mixed land-sea path correction reduces the median basic transmission loss. 

To complete the discussion of the extended Hata model, note that [A-1] also provided estimates 

of the standard deviation of the location variability for urban and suburban environments in [A1] Figure 39. A version of these curves suitable for computation/extrapolation is required. Noting 

that the frequency dependence is non-linear in log , one tries a fit of the form: αα + ββ log +

γγ(log )2 using the values in Table A-1 that were extracted from the curves.

Table A-1. Okumura et al. Location Variability Standard Deviations

f (MHz) σσuu (dB) σσrr (dB)

1500 7.1 8.95

2000 7.5 9.5

3000 8.1 10.1

These values yield the following three simultaneous equations for σσuu in the unknowns ααuu, ββuu

and γγuu:

7.1 = ααuu + ββuu log(1500) + γγuu(log(1500))2 (A-18a)

7.5 = ααuu + ββuu log(2000) + γγuu(log(2000))2 (A-18b)
35

8.1 = ααuu + ββuu log(3000) + γγuu(log(3000))2 (A-18c)

which evaluate to:

ααuu = 4.0976291,

ββuu = -1.2255656,

γγuu = 0.68350345.

Note also that it is probably sufficient to set σσrr = σσuu + 2.

A.4 The Irregular Terrain Model (ITM)

The ITS Irregular Terrain Model (ITM) is a general purpose radio propagation prediction 

algorithm intended for use on tropospheric radio circuits utilizing frequencies in the range 

20 MHz–20 GHz. The model is based on electromagnetic theory and statistical analyses of 

terrain features and radio measurements. The model predicts the median signal strength as a 

function of distance and the variability of the signal strength in time (because of changing 

atmospheric conditions) and space (because of changes in terrain). The model does not include 

the fine details of channel characterization (e.g., fast fading).

This description of the ITM algorithm focuses first on the computation of the reference 

attenuation, that is, the median computed attenuation relative to free space, and then on a 

discussion of the how the cumulative distributions of attenuation are obtained. This description 

covers the use of the model in its point-to-point mode, although mention is made of area mode 

predictions. The distinction between the use of the model in the area prediction mode and the 

point-to-point prediction mode is chiefly in the additional input parameters/data that must be 

provided, generally through examination of the detailed terrain profile between the two 

terminals. Automated procedures for examining the detailed terrain profile to obtain the 

additional inputs are available.

The input parameters in Table A-2 are required to use ITM in either mode.

Table A-2. Irregular Terrain Model Input Parameters/Data and Preliminary Calculations

dd the (horizontal) distance between the terminals [m]

hgg1, hgg2 the terminals’ antenna structural (i.e., electrical center) heights above ground [m]

kk the wavenumber [m-1

]

∆h the terrain irregularity parameter (i.e., the inter-decile range of terrain elevations on the 

path) [m]

ss the minimum monthly mean surface refractivity [N-units]

γγ the earth’s effective curvature [m-1

]

ZZgg the surface transfer impedance of the ground

one of seven discrete radio climate types
36

The wavenumber, k, is given in terms of the wavelength in a vacuum, λ, by:

kk = 2ππ

λλ = 2ππ (A-19)

where the speed of light in a vacuum, = 1

�μμ0εε0

≅ 2.997 × 108 m s-1. If one measures the 

frequency, , in units of MHz, rather than Hz, then it is convenient to write the wavenumber as:

kk = 0

(A-20)

where 0 = 47.7 MHz m.

The surface refractivity, ss, is sometimes given in terms of the surface refractivity reduced to 

sea-level, 0. If the surface refractivity is specified as its value at sea level, then it should be 

adjusted to account for the general elevation of the region involved, zzssssss [m]:

ss = 0

− zzssssss

9.46×103 (A-21)

The earth’s effective curvature, γγ [m-1], is determined from the surface refractivity, ss, using 

the empirical formula from [A-4] (4.4):

γγ = γγ �1 − 0.04665

NNss

NN1� = γγ

KK

(A-22)

where KK is the “effective earth’s radius factor”, γγ = ππ

2

× 10−7 m-1 is the physical earth’s 

curvature, and 1 = 179.3 N-units. The effective earth’s radius, , is given by:

= γγ −1 = KK (A-23)

where the physical earth’s radius, , is given by = γγ −1 ≅ 6.37 × 106.

The surface transfer impedance of the ground is a complex, dimensionless constant which 

depends on the relative dielectric permittivity of the ground, εεrr = εε

εε0

, and electrical conductivity, 

of the ground, σσ, and the polarization of the radio signal’s electric field vector (i.e., horizontal, 

which is defined as perpendicular to the plane of incidence at the ground reflection point, or 

vertical, which is defined as parallel to the plane of incidence at the ground reflection point). The 

complex relative dielectric permittivity, εε′, is given by:

εε′ = εεrr +

σσ

kk �

μμ0

εε0

(A-24)

and, then, the surface transfer impedance of the ground, ZZgg, is
37

ZZgg = �

√ε′ − 1 for horizontal polarization

√ε′ − 1

ε′ for vertical polarization

(A-25)

For ITM in area prediction mode, the siting criteria for the terminals are required. These are 

qualitative measures of the care taken, at each terminal, to attempt to ensure good radio 

propagation conditions. There are three levels: at random, with care, and with great care.

For ITM in point-to-point prediction mode, a detailed terrain elevation profile must be provided 

at equally spaced horizontal distance increments between the two terminals. The sequence of 

elevations and their corresponding distances is then used to extract the terminals’ irregular 

terrain radio horizon distances, ddll1, ddll2, the terminals’ irregular terrain radio horizon elevation 

angles, θθ 1, θθ 2, the terminals’ effective heights, h 1, h 2, and the terrain irregularity parameter, 

∆h.

The “effective height” of an antenna is its height above an “effective reflecting plane” or above 

the “intermediate foreground” between the antenna and its horizon. A difficulty with the model 

is that there is no explicit, quantitative, definition of this quantity, and the accuracy of the model 

sometimes depends on the skill of the user in estimating values for these effective heights. The 

terminals’ effective antenna heights determine the terminals’ smooth earth radio horizon 

distances. The ITM automated procedures are used in this analysis to compute the terminals’ 

effective heights. Another user might override these procedures and, instead, specify these values 

based on his/her own educated estimates.

In ITM’s point-to-point prediction mode, the terminals’ irregular terrain radio horizon distances 

and elevation angles are found by geometric methods (see, e.g., [A-4]), given the line drawn 

between the two terminals’ electrical centers and the terrain elevations’ heights with respect to 

that line. The profile is assumed to lie along the great circle path containing the terminals and the 

curvature of the great circle path is γγ , so that the effective loss of antenna height due to earth 

curvature through arc-length dd is approximately − γγeedd2

2

. If none of the intermediate terrain 

elevations is higher than the line between the two terminals’ electrical centers, then the path is 

line-of-sight. Otherwise, the path is trans-horizon.

In ITM’s point-to-point prediction mode, this distinction between line-of-sight and trans-horizon 

paths is important. If the path is line-of-sight, then no horizons are detected during examination 

of the terrain profile. However, the model requires that these quantities be defined, so, for lineof-sight paths in the point-to-point mode only, the terminals’ irregular terrain radio horizon 

distances and elevation angles are determined using the formulas from the area mode which are 

given below. In using the area mode formulas, it may happen that the point-to-point mode path 

distance, dd > ddll = ddll1 + ddll2, which, in turn, implies that the path is trans-horizon. To remedy 

this contradiction, it suffices to conjecture that the terminals’ effective heights have been 

underestimated, and then these effective heights are increased by the common factor that yields 

dd ≤ ddll.

In the area prediction mode, the model does not require a terrain profile, but it should be 

emphasized that this does not mean that the model’s area prediction mode does not account for 
38

irregular terrain. Instead, the terminals’ radio horizon distances, irregular terrain horizon

elevation angles, and effective heights are determined by the terminals’ antenna heights above 

ground, their siting criteria and the terrain irregularity parameter. This last quantity is user 

supplied, either from estimates based on many terrain profiles covering the region involved or 

from typical, suggested values ([A-5] Table 2.).

Beginning with the determination of the terminals’ effective heights in the area prediction mode 

exclusively, if the terminal in question is sited at random, then:

h jj = hggjjfor jj = 1 oo 2 (A-26)

Otherwise, and again, exclusively in the area prediction mode, let

BBjj = � 5 if terminal j is sited with care

10 if terminal j is sited with great care

(A-27a)

and

BBjj

′ = �BBjj − 1� sin�

ππ

2 �

hggjj

5 , 1�� + 1

(A-27b)

Note that BBjj

′ = BBjj if hggjj ≥ 5 . The j

th terminal’s effective height, h jj(jj = 1 oo 2), is then 

h jj = hggjj + BBjj′ −

2hggjj

∆h

(A-27c)

In the area prediction mode (and in the point-to-point mode for line-of-sight paths), the 

terminals’ observed median horizon distances and irregular terrain horizon elevation angles are 

then determined from their corresponding effective heights [A-6], the effective earth’s curvature 

and the terrain irregularity parameter. First, one computes the smooth earth horizon distances, 

ddll jj (jj = 1,2):

ddll jj = �

2h jj

γγ

(A-28)

and then the terminals’ irregular terrain radio horizon distances, ddlljj (jj = 1,2), and elevation 

angles, θθ jj (jj = 1,2), are given by

ddlljj = ddll jj

−0.07� ∆h

mm �heejj,5�

(A-29)
39

(Note: in point-to-point mode for line-of-sight paths if dd > ddll = ddll1 + ddll2, then set h jj =

�dd

ddll

�

2

h jj (jj = 1,2) and re-compute the smooth earth and irregular terrain radio horizon 

distances.)

Then set

θθ jj = −

�2h jj + 0.65∆h �1 −

0.07� ∆h

mm �heejj,5�

��

ddll jj

(A-30)

In either the area or the point-to-point mode, the model requires the combined smooth earth radio 

horizon distance, ddll , the combined irregular terrain radio horizon distance, ddll, and the combined 

irregular terrain radio horizon elevation angles, θθ . From (A-28),

ddll jj = �

2h jj

γγ jj = 1,2

ddll = ddll 1 + ddll 2 (A-31)

ddll = ddll1 + ddll2 (A-32)

θθ = �θθ 1 + θθ 2, −ddllγγ � (A-33)

and the definition of a function of a distance, s, which asymptotically approaches the terrain 

irregularity parameter (see [A-6]):

∆h( ) = ∆h �1 − 0.8 − ss

5×104� (A-34)

A.4.1 The Reference Attenuation 

The reference attenuation, AArrrr , is given as a piecewise function of the path horizontal distance, 

dd, by:

AArrrr =

⎩

⎨

⎧ �0, AA + KK1dd + KK2 ln � dd

ddll

�� for dd ≤ ddll

AA + dddd for ddll ≤ dd ≤ dd

AA + ssdd for dd ≤ dd

(A-35)

where the coefficients AA ,KK1,KK2, AA , dd, AA , ss, and the tropospheric scatter cross-over 

distance, dd , are given in the following. The three distance ranges/intervals given above are 
40

referred to as the line-of-sight range, the diffraction range and the scatter range, respectively. The 

function is continuous, so at the second interval’s endpoints, dd = ddllss, dd , respectively, the lower 

and higher distance intervals’ expressions are required to yield equal values for AArrrr . Therefore, 

there are only five independent coefficients.

A.4.2 The Coefficients of the Diffraction Range

The coefficients for the diffraction range are found by evaluating the diffraction 

attenuation,AAdddd (dd), at two distances, dd3 and dd4, which are beyond line-of-sight. Set

dd3 = (ddll , ddll + 1.3787XX ) (A-36)

dd4 = dd3 + 2.7574XX (A-37)

AA3 = AAdddd (dd3) (A-38)

AA4 = AAdddd (dd4) (A-39)

where

XX = (kkγγ2)

−1

3 = �

2ππ

λλ �

−1

3 (A-40)

Then the desired coefficients are given by:

dd = AA4 − AA3

dd4 − dd3

(A-41)

AA = AA3 − dddd3 = AA3dd4 − AA4dd3

dd4 − dd3

(A-42)

A.4.3 The Diffraction Range Attenuation Function

The diffraction attenuation function is a weighted combination of both knife-edge and roundedearth diffraction attenuations plus a terrain “clutter” attenuation:

AAdddd ( ) = �1 − ww( )�AAkk( ) + ww( )AArr( ) + AAffff (A-43)

The idea is to design the weighting factor, ww( ), in such a way that it is unity when the distance 

dependent terrain irregularity parameter, Δh( ), vanishes and that the weighting factor vanishes 

when the distance dependent terrain irregularity parameter becomes large. To achieve this result, 

define
41

ww( ) = 1

1 + 0.1�QQ( )

(A-44)

with

QQ( ) = �

Δh( )

λλ , 1000� ��

h 1h 2 + CC

hgg1hgg2+ CC +

ddll + θθ �

(A-45)

and

CC = � 0 2 in area prediction mode

10 2 in point − to − point mode

(A-46)

In the diffraction range attenuation function, AAdddd ( ), the additional terrain “clutter” 

attenuation, AAffff, is intended to approximately account for the median additional diffraction 

attenuation due to additional knife-edges between the terminals’ irregular terrain radio horizons 

that may obstruct the convex hull between the two irregular terrain radio horizons. Its median 

value is empirically determined and is given by

AAffff = �15, 5 log �1 + 4.77 × 10−4kkhgg1hgg2σσh(ddll )�� (A-47)

where

σσh(ddll ) = ∆h(ddll )

1.282

− �∆h(ddll ) 4

2

(A-48)

The median double knife-edge diffraction attenuation for the two irregular terrain radio horizons 

is then given by a double knife-edge Epstein-Peterson like [A-7] approximation:

AAkk( ) = �νν1( )� + �νν2( )� (A-49)

with

(νν) = 20 log �� 1

√2 � ππ

2uu2

dd

∞

νν

��

(A-50)

and

ννjj( ) = θθ( )

2 �

2ddlljj

( − ddll)

λλ � − ddll + ddlljj�

for jj = 1,2

(A-51)

where
42

θθ( ) = θθ + γγ (A-52)

The rounded-earth diffraction attenuation is based on a “three radii” method applied to Vogler’s

formulation of the solution of the smooth spherical earth diffraction problem [A-8]. The three 

curvatures, i.e., the inverses of the three radii, are given by:

γγ0 = θθ( )

− ddll

(A-53a)

and

γγjj = 2h jj

ddlljj

2 for jj = 1,2

(A-53b)

Next, set

ααjj = �

kk

γγjj

�

1

3

for jj = 0,1,2

(A-54)

and

Κjj = 1

ααjjZZgg

for jj = 0,1,2 (A-55)

In the next step, define the dimensionless distances:

xxjj = AA �Κjj�ααjjγγjjddlljj for jj = 1,2 (A-56a)

and

xx0 = AA (Κ0)αα0γγ0( − ddll) + xx1 + xx2

= AA (Κ0)αα0θθ( ) + xx1 + xx2

(A-56b)

where AA is a dimensionless constant equal to 151.03. Of the dimensionless distances defined 

above, note that only xx0 depends on . The rounded-earth diffraction attenuation, AArr( ), is given 

by:

AArr( ) = GG(xx0) − (xx1, Κ1) − (xx2, Κ2) − CC1(Κ0) (A-57)

where the functions GG(xx), (xx, Κ), CC1(Κ) and BB(Κ) are defined by [A-8].

A.4.4 The Coefficients of the Line-of-Sight Range

As noted above, in the line-of-sight range, the idea is to devise a curve for the reference 

attenuation of the form:
43

AArrrr = AA + KK1dd + KK2 ln � dd

ddll

� (A-58)

with the added constraints that KK1,KK2 ≥ 0 and that continuity is ensured at the endpoint of the 

interval. To ensure continuity, set:

dd2 = ddll (A-59)

and

AA2 = AA + dddd2 (A-60)

There are two general cases, depending upon the sign of AA .

First, consider the general case AA ≥ 0. To determine the coefficients, the attenuations at two 

additional distances within the line-of-sight range are required, so set:

dd0 = �

ddll

2 , 1.908kkh 1h 2� (A-61)

dd1 = 3

4 dd0 +

ddll

4

(A-62)

AA0 = AAll (dd0) (A-63)

AA1 = AAll (dd1) (A-64)

Solving simultaneously first for the unknowns KK1,KK2 yields the provisional values:

KK2

′ = �0,

(AA1 − AA0)(dd2 − dd0) − (AA2 − AA0)(dd1 − dd0)

(dd2 − dd0) ln �

dd1

dd0

� − (dd1 − dd0) ln �

dd2

dd0

�

�

(A-65)

KK1

′ =

AA2 − AA0 − KK2

′ ln �

dd2

dd0

�

dd2 − dd0

(A-66)

If KK1

′ ≥ 0, then the added constraints are satisfied, so set:

KK1 = KK1

′ (A-66a)

KK2 = KK2

′ (A-66b)

However, if KK1

′ < 0, then more work is required so, provisionally discard KK1,AA1, which yields a 

pure power-law dependence with distance in the line-of-sight range:
44

KK2′′ = AA2 − AA0

ln �

dd2

dd0

�

(A-67)

If KK2′′ ≥ 0, then

KK1 = 0 (A-68a)

KK2 = KK2′′ (A-68b)

If, on the other hand, KK2′′ < 0, then AA0 > AA2, so discard both AA0, AA1and set

KK1 = dd (A-69a)

KK2 = 0 (A-69b)

This last alternative corresponds to extrapolating the diffraction range attenuation back into the 

line-of-sight range and, given that AA ≥ 0, ensures that the reference attenuation will always be 

greater than or equal to zero.

Now consider the second general case: AA < 0. As before, the attenuations at two additional 

distances in the line-of-sight range are required. As a starting point, set

dd0 = 1.908kkh 1h 2 (A-70)

dd1 = �− AA

dd

,

ddll

4 � (A-71)

Note that the first of the two distances in the comparative expression for dd1is the distance at 

which the extrapolated diffraction range attenuation vanishes. The procedure for computing the 

unknown coefficients is now a bit more involved than before, because there is no a priori 

guarantee that dd0 < dd1.

If dd0 < dd1, then, as before, begin by attempting to use all three distances. Set

AA0 = AAll (dd0) (A-72)

AA1 = AAll (dd1) (A-73)

and then solve simultaneously for provisional values of KK1,KK2, as before:

KK2

′ = �0,

(AA1 − AA0)(dd2 − dd0) − (AA2 − AA0)(dd1 − dd0)

(dd2 − dd0) ln �

dd1

dd0

� − (dd1 − dd0) ln �

dd2

dd0

�

�

(A-74)

If KK2

′ = 0, then immediately branch to the dd0 ≥ dd1case given below. Otherwise, evaluate
45

KK1

′ =

AA2 − AA0 − KK2

′ ln �

dd2

dd0

�

dd2 − dd0

(A-75)

Now, if KK1

′ ≥ 0 (and since KK2′ > 0), the constraints are satisfied:

KK1 = KK1

′ (A-76a)

KK2 = KK2

′ (A-76b)

However, if KK1

′ < 0, then more work is required so discard KK1, AA1 (which, as before, yields a 

pure power-law dependence with distance in the line-of-sight range):

KK2′′ = AA2 − AA0

ln �

dd2

dd0

�

(A-77)

If KK2′′ ≥ 0, then

KK1 = 0 (A-78a)

KK2 = KK2′′ (A-78b)

If, on the other hand, KK2′′ < 0, then AA0 > AA2, so discard both AA0, AA1and set:

KK1 = dd (A-79a)

KK2 = 0 (A-79b)

If dd0 ≥ dd1or KK2

′ = 0, then discard AA0:

AA1 = AAll (dd1) (A-80)

KK1

′′ = AA2 − AA1

dd2 − dd1

(A-81)

Now, if KK1′′ > 0, then

KK1 = KK1′′ (A-82a)

KK2 = 0 (A-82b)

Otherwise, it becomes necessary to discard both AA0, AA1:

KK1 = dd (A-83a)

KK2 = 0 (A-83b)

To complete the calculation of the line-of-sight range coefficients, note that it is sufficient to set
46

AA = AA2 − KK1dd2 (A-84)

A.4.5 The Line-of-Sight Range Attenuation Function

The line-of-sight range attenuation function, AAll ( ), is given by a weighted combination of the 

“extrapolated/extended diffraction range attenuation”, AAdd( ), and the “two-ray attenuation”, 

AAtt( ):

AAll ( ) = (1 − ww)AAdd( ) + wwAAtt( ) (A-85)

The idea here is to devise a weighting function, ww, which varies from unity as the terrain 

irregularity parameter, ∆h, vanishes and down to zero as ∆h becomes large compared to the 

wavelength:

ww = 1

1 + 47.7kk∆h

(DD2, ddll )

(A-86)

where DD2 = 10 km. The extended diffraction range attenuation is given by:

AAdd( ) = AA + dd (A-87)

To obtain the two-ray attenuation, it is necessary to obtain an estimate of the effective reflection 

coefficient of the ground plane. Set

sin ψψ( ) = h 1 + h 2

� 2 + �h 1 + h 2�

2

(A-88)

and then set the provisional effective reflection coefficient to the product of the Fresnel reflection 

coefficient (for small grazing angles) and a correction for surface roughness in the first Fresnel 

zone of the ground reflection point based on the Rayleigh criterion

′ ( ) = sin ψψ( ) − ZZgg

sin ψψ ( ) + ZZgg

−kkσσh(ss) sin ψψ(ss) (A-89)

where

σσh( ) = ∆h( )

1.282

− �∆h(ss) 4

2

(A-90)

Also, provisionally approximate the phase difference due to the path length differences between 

the direct and ground-reflected rays as
47

δδ′

( ) ≈ 2

kkh 1h 2 (A-91)

In ITM, the effective reflection coefficient, ( ), is given by

( ) = �

′ ( ) for | ′( )| ≥ �0.5, �sin ψψ( )�

′( )

| ′( )|

�sin ψψ( ) otherwise

(A-92)

The phase difference, δδ( ), is then given by

δδ( ) =

⎩

⎪

⎨

⎪

⎧ δδ′

( ) for δδ′( ) ≤

ππ

2

ππ − �

ππ

2�

2

δδ′( ) otherwise

(A-93)

which corrects for the approximation when the distance, , is small.

The two-ray attenuation, AAtt( ), is then given by:

AAtt( ) = −20 log�1 + ( ) ii(ss)� (A-94)

which is a good approximation as long as the distance, s, is not too small (i.e., the direct ray’s 

path length and the ground reflected ray’s path length are nearly equal).

A.4.6 The Coefficients of the Forward (or Tropospheric) Scatter Range

To find the coefficients of the scatter range, first set:

dd5 = ddll + DDss (A-95)

dd6 = dd5 + DDss (A-96)

with DDss = 200 km and then compute the tropospheric scatter attenuations at these distances

AA5 = AAssss (dd5) (A-97)

AA6 = AAssss (dd6) (A-98)

If both AA5 and AA6 are defined, then

ss = AA6 − AA5

DDss

(A-99)
48

dd = �ddll , ddll + XX log(47.7kk),

AA5 − AA − ssdd5

dd − ss

� (A-100)

AA = AA + ( dd − ss)dd (A-101)

If either AA5 or AA6 is undefined, then set

dd = ∞ (A-102)

A.4.7 The Scatter Range Attenuation Function 

This function is computed using modified and abbreviated methods for computing the forward 

scatter loss found in [A-4] in Chapter 9 of Volume I and Appendix III.5 of Volume II. The 

following geometric parameters are set

θθ = θθ + γγ (A-103)

θθ′ = θθ 1 + θθ 2 + γγ (A-104)

jj = 2kk ′h jj for jj = 1,2 (A-105)

If both 1 and 2 are less than 0.2, then the function AAssss is undefined (or infinite). For other 

combinations of the values of 1and 2, the function is given by:

AAssss ( ) = 10 log(47.7kkθθ4) + (θθ , ss) + 0 (A-106)

where the attenuation function, (θθ , ss), and the “frequency gain” function, 0, are given 

below.

To obtain the attenuation function, (θθ , ss), it is useful to note that Rice et al. [A-4] give the 

reference value, LLbb , of the long-term median basic transmission loss due to forward scatter ([A4] (9.1)) as:

LLbb = 30 log − 20 log dd + (θθ ) − 0 + 0 + AA (A-107)

where is the frequency (MHz), dd is the sea-level arc-length distance (km) and θθ is the angular 

distance (radians). Neglecting the last three terms on the right-hand side of (A-107), the 

reference long-term median attenuation relative to free space due to forward scatter is then

LLbb − LLff = 30 log − 20 log dd + (θθ ) − 20 log(2kk ) (A-108)

where = dd × 103. After some manipulation, it can be shown that

LLbb − LLff = 10 log(47.7kkθθ4) − 32.44 + (θθ ) − 40 log(θθ ) (A-109)

with
49

(θθ ) = �

135.82 + 0.332θθ + 30 log(θθ ) 0.01 ≤ θθ ≤ 10

129.5 + 0.212θθ + 37.5 log(θθ ) 10 ≤ θθ ≤ 70

119.2 + 0.157θθ + 45 log(θθ ) otherwise

(A-110)

for ss = 301 (see [A-4], Volume II, (III.46)–(III.48)). Absorbing the last three terms on the 

right-hand side of (A-109) into the sought after attenuation function, (θθ , ss), with DD ≡ θθ =

θθ × 103, yields:

(DD, 301)

= �

133.4 + 0.332 × 10−3DD − 10 log(DD) 0 ≤ DD ≤ 104

104.6 + 0.212 × 10−3DD − 2.5 log(DD) 104 ≤ DD ≤ 7 × 104

71.8 + 0.157 × 10−3DD + 5 log(DD) otherwise

(A-111)

and

(DD, ss) = (DD, 301) − 0.1( ss − 301) − DD

4×104 (A-112)

The “frequency gain” function, 0, is a function of 1, 2, the scatter efficiency factor ηηss, and the 

asymmetry factor, ss, of the cross-over point. The cross-over point is assumed to be midway 

between the terminals’ radio horizons, so the asymmetry factor, ss, is then given by:

ss = ddll2 + � − ddll1 − ddll2�

2

ddll1 + � − ddll1 − ddll2�

2

= − �ddll1 − ddll2�

+ �ddll1 − ddll2�

(A-113)

If it happens that the asymmetry factor, as defined above, is greater than unity, then two 

adjustments are made. Firstly, the asymmetry factor, ss, is set to its reciprocal. Secondly, the 

ratio of the terminals’ effective heights, �

rr2

rr1

�, is also set to its reciprocal. The height of the crossover point, zz0, is:

zz0 = ′ ss

(1 + ss)2

(A-114)

and the scatter efficiency factor, ηηss, is

ηηss = zz0

ZZ0

�1 + �3.1 × 10−2 − 2.32 × 10−3 ss + 5.67

× 10−6 ss

2� −�

zz0

ZZ1

�

6

�

(A-115)

with ZZ0 = 1.756 × 103 m and ZZ1 = 8.0 × 103 m. The frequency gain function, 0, is given by

0 = 00( 1, 2, ηηss) + Δ 0( ss, qq, ηηss) (A-116)
50

where 

qq =

� 2

1

�

ss

(A-117)

Δ 0( ss, qq, ηηss) = 6(0.6 − log ηηss) log ss log qq (A-118)

The function 00 is obtained by linear interpolation on integer values of the scatter efficiency, 

ηηss. For ηηss = jj = 1, ... ,5 the function takes the form:

00( 1, 2,jj) = 1

2 � 01( 1,jj) + 01( 2,jj)� (A-119)

with

01( , 1) = 10 log �1 +

24

2 +

25

4 � (A-120a)

01( , 2) = 10 log �1 +

45

2 +

80

4 � (A-120b)

01( , 3) = 10 log �1 +

68

2 +

177

4 � (A-120c)

01( , 4) = 10 log �1 +

80

2 +

395

4 � (A-120d)

01( , 5) = 10 log �1 +

105

2 +

705

4 � (A-120e)

For ηηss > 5, the value at ηηss = 5 is used. For ηηss = 0,

00( 1, 2, 0) = 10 log ��1 +

√2

1

�

2

�1 +

√2

2

�

2

1 + 2

1 + 2 + 2√2

�

(A-121)

For all of this computation the values of ss and q are truncated at 0.1 and 10.0.

A.4.8 Variability and the Quantiles of the Attenuation

The preceding discussion provided a method for computing the reference attenuation as a 

function of distance. Given the input quantities, the method is apparently deterministic, albeit 

with scattered empiricisms. However, as pointed out in [A-5] “it seems undeniable that received 

signal levels are subject to a wide variety of random variations and that proper engineering must 

take these variations into account.” In order to take these variations into account, ITM provides a 

simple model which uses random variables, each of which depends on only one of the three 

dimensions of variability: time, locations and situations.
51

When the signal levels are expressed in decibel notation, experience shows that the distributions 

involved are normal or nearly normal. Therefore, the calculations of the desired fractions (or 

quantiles) of the attenuation, AA(qqTT, qqLL, qqSS), for time, qqTT, locations, qqLL, and situations, qqSS, are 

rescaled in terms of “standard normal deviates”.

Following the convention of [A-4], Volume II, Annex V (see also, e.g., [A-5], Section 6.2), a 

positive deviation corresponds to an increase in signal level and, so, to a decrease in attenuation 

or loss. The complementary normal distribution for the value, zz, which is exceeded for a given 

fraction (i.e., with probability), QQ:

QQ(zz) = 1

√2ππ

� −tt2

2 dd

∞

zz

(A-122)

with its inverse function giving the standard normal deviate, zz, is given by:

zz(qq) = QQ−1(qq) (A-123)

Thus one can say that if a random variable, xx, is normally distributed with mean (and median) 

value, XX0, and standard deviation, σσ, then its quantiles are given by

XX(qq) = XX0 + σσ (qq) (A-124)

Now set:

zzTT = zz(qqTT) (A-125a)

zzLL = zz(qqLL) (A-125b)

zzSS = zz(qqSS) (A-125c)

and ask for the quantile of attenuation, AA(zzTT, zzLL, zzSS). Given the sign convention for the 

deviations, let the provisional value of this quantile be:

AA′ ≡ AArrrr − VVmm − YYTT − YYLL − YYSS (A-126)

with VVmm , YYTT, YYLL and YYSS defined below. The deviations, YYTT , YYLL and YYSS, have zero mean (and 

median) values. 

To avoid predicting basic transmission loss values very much less than the free space loss, set:

AA(zzTT, zzLL, zzss) = �

AA′ if AA′ ≥ 0

AA′ 29 − AA′

29 − 10AA′

otherwise

(A-127)
52

A.4.8.1 The Effective Distance

The “effective distance” is important in the evaluation of the median adjustment, VVmm , and the 

deviations associated with the time variability, YYTT, and the situation variability, YYSS. The transition 

distance, dd , is given by:

dd = �2 1h 1 + �2 1h 2 + 1(kkDD1)

−1

3 (A-128)

with 1 = 9000 km and DD1 = 1266 km. The effective distance, dd , is a piecewise function of 

distance:

dd = �dd

DD0

dd for dd ≤ dd

DD0 + dd − dd for dd > dd

(A-129)

with DD0 = 130 km. Note that the effective distance depends on the terminals’ effective antenna 

heights and the frequency through dd . For example, with h 1 = 50 m, h 2 = 6 m and =

3500 MHz, dd ~60.3 km.

A.4.8.2 The All Year Median Adjustment 

The computed reference attenuation, AArrrr , is adjusted by the all year median, 

VVmm (dd , ), to obtain the median quantile of attenuation. This adjustment is shown in 

Figure 10.13 of [A-4]. For effective distances less than ~80 km, the adjustment is less than 1 dB 

for all radio climates. For the maritime temperate oversea radio climate, the maximum 

adjustment is ~7 dB at an effective distance of ~220 km, falling gradually to ~3 dB at great 

effective distances.

A.4.8.3 The Deviation of Time Variability 

The deviation of the time variability, YYTT, is piecewise linear in zzTT. It takes the form of:

YYTT = �

σσTT−zzTT zzTT ≤ 0

σσTT+zzTT 0 ≤ zzTT ≤ zzDD

σσTT+zzDD + σσTT (zzTT − zzDD) zzDD ≤ zzTT

(A-130)

As is the case with the all year median adjustment, the “fading” and “enhancement” pseudostandard deviations, σσTT− and σσTT+ respectively, are both functions of the effective distance and 

the radio climate:

σσTT− = σσTT−(dd , ) (A-131)

σσTT+ = σσTT+(dd , ) (A-132)
53

These pseudo-standard deviations are found by taking the curves shown in [A-4] Volume I, 

Section 10 and Volume II, Section III.7, and dividing the YY(0.9) and YY(0.1) long term power 

fading deviation quantiles by their corresponding standard normal deviates, −1.282 and 1.282, 

respectively.

For the low probability or “ducting” case (i.e., zzDD ≤ zzTT), there are two additional constants that 

depend on the radio climate:

zzDD = zzDD( ) (A-133)

σσTT = CCDD( )σσTT+ (A-134)

Values for these constants are given in Table A-3 along with qqDD = QQ(zzDD).

Table A-3. Longley-Rice Low Probability or “Ducting” Constants

Radio Climate qqDD zzDD CCDD

Equatorial 0.10 1.282 1.224

Continental Subtropical ~ 0.015 2.161 0.801

Maritime Subtropical 0.10 1.282 1.380

Desert 0.00 ∞ -

Continental Temperate 0.10 1.282 1.224

Maritime Temperate Overland 0.10 1.282 1.518

Maritime Temperate Oversea 0.10 1.282 1.518

A.4.8.4 The Deviation of Location Variability 

The deviation of the location variability, YYLL, is given by:

YYTT = σσLLzzLL (A-135)

with the standard deviation of location variability, σσLL, given by:

σσLL = 10kk∆h(dd)

13 + kk∆h(dd)

(A-136)

where dd is the path length (i.e., distance). If the terrain irregularity parameter for this path 

distance is more than just a few wavelengths in size, then σσLL~10 dd . Of course, if both terminals 

are at known, fixed positions and the model is being used in point-to-point mode, then zzLL = 0.

A.4.8.5 The Deviation of Situation Variability 

The standard deviation of the situation variability, σσSS, is given by ([A-6], Annex 1, (1.16)):
54

σσSS = 5 + 3 − ddee

105 (A-137)

The deviation of the situation variability, YYSS, depends on the preceding quantity and terms 

dependent on the deviations of the time and location variabilities which account for the 

convolution of the three mutually independent random variables:

YYSS = zzSS�σσSS

2 +

YYTT

2

7.8 + zzSS

2 +

YYLL

2

24 + zzss

2

(A-138)

(For a discussion, see, e.g., [A-5] Section 6.2.)

A.5 References

[A-1] Y. Okumura, E. Ohmori, T. Kawano, and K. Fukuda, “Field strength and its variability in 

VHF and UHF land-mobile radio service, Rev. Elec. Commun. Lab., 16, 9-10, pp. 825-

873, (Sept.-Oct. 1968).

[A-2] M. Hata, “Empirical formula for propagation loss in land mobile radio services,” IEEE 

Transactions on Vehicular Technology, VT-29, 3, pp. 317-325 (Aug. 1980).

[A-3] A.G. Longley, Radio Propagation in Urban Areas, U.S. Department of Commerce, 

Office of Telecommunications, OT Report 78-144 (Apr.1978), available at 

http://www.its.bldrdoc.gov/publications/2674.aspx. 

[A-4] P.L. Rice, A.G. Longley, K.A. Norton, and A.P. Barsis, Transmission Loss Predictions 

for Tropospheric Communication Circuits, U.S. Department of Commerce, National 

Bureau of Standards, Technical Note 101 (Revised January 1, 1967), Vol. I available at 

http://www.its.bldrdoc.gov/publications/2726.aspx, Vol. II available at 

http://www.its.bldrdoc.gov/publications/2727.aspx.

[A-5] G.A. Hufford, A.G. Longley, and W.A. Kissick, A Guide to the Use of the ITS Irregular 

Terrain Model in Area Prediction Mode, National Telecommunications and Information 

Administration Report 82-100 (April 1982), available at http://www.its.bldrdoc.gov/

publications/2091.aspx.

[A-6] A.G. Longley and P.L. Rice, Prediction of Tropospheric Radio Transmission Loss Over 

Irregular Terrain: A Computer Method–1968, U.S. Department of Commerce, 

Environmental Science Services Administration, ESSA Technical Report ERL 79-ITS 67 

(July 1968), available at http://www.its.bldrdoc.gov/publications/2784.aspx.

[A-7] J. Epstein and D.W. Peterson, “An experimental study of wave propagation at 850 Mc,” 

Proc. IRE 41, pp. 595-611 (1953).
55

[A-8] L. E. Vogler, “Calculation of groundwave attenuation in the far diffraction region,” Radio 

Science Journal of the Research NBS/USNC-URSI, Vol. 68D, No. 7, pp. 819-826 (July 

1964).
56

APPENDIX B SHIPBORNE RADAR ANALYSIS RESULTS

B.1 Introduction

This appendix provides the results of the exclusion zone distance analysis and the exclusion 

zones needed to protect the shipborne radars operating in the 3550–3650 MHz band. 

B.2 Shipborne Radar Analysis Results

The initial analysis results indicated that the shipborne radar systems operating below 3550 

MHz, Shipborne Radar 2 and Shipborne Radar 3, did not require exclusion zones for protection. 

The exclusion zone distance analysis results presented in this appendix are for Shipborne 

Radar 1, Shipborne Radar 4, and Shipborne Radar 5. The results are shown on Figures B-1

through B-9 for Shipborne Radar 1; Figures B-10 through B-18 for Shipborne Radar 4; and 

Figures B-19 through B-27 for Shipborne Radar 5. The yellow line on each plot is the exclusion 

zone distance based on the Fast Track Report and the blue line is the revised exclusion zone 

distance. The red and blue dots in the figures represent the ship location off-shore.

Figure B-1. Shipborne Radar 1–exclusion zone lower 48 states (yellow line–fast track exclusion 

zone and blue line–revised exclusion zone).
57

Figure B-2. Shipborne radar 1–Exclusion zone upper west coast (yellow line–fast track exclusion 

zone and blue line–revised exclusion zone).

Figure B-3. Shipborne Radar 1–Exclusion zone middle west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
58

Figure B-4. Shipborne Radar 1–Exclusion zone lower west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-5. Shipborne Radar 1–Exclusion zone upper east coast (yellow line–fast track exclusion 

zone and blue line–revised exclusion zone).
59

Figure B-6. Shipborne Radar 1–Exclusion zone middle east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-7. Shipborne Radar 1–exclusion zone lower east coast (yellow line–fast track exclusion 

zone and blue line–revised exclusion zone).
60

Figure B-8. Shipborne Radar 1–Exclusion zone eastern gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-9. Shipborne Radar 1–Exclusion zone western gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
61

Figure B-10. Shipborne Radar 4–Exclusion zone lower 48 states (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone). 

Figure B-11. Shipborne Radar 4–Exclusion zone upper west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
62

Figure B-12. Shipborne Radar 4–Exclusion zone middle west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-13. Shipborne Radar 4–Exclusion zone lower west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
63

Figure B-14. Shipborne Radar 4–Exclusion zone upper east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-15. Shipborne Radar 4–Exclusion zone middle east coast(yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
64

Figure B-16. Shipborne Radar 4–exclusion zone lower east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-17. Shipborne Radar 4–Exclusion zone eastern gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
65

Figure B-18. Shipborne Radar 4–Exclusion zone eastern gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-19. Shipborne Radar 5–Exclusion zone lower 48 states(yellow line–fast track exclusion 

zone and blue line–revised exclusion zone). 
66

Figure B-20. Shipborne Radar 5–Exclusion zone upper west coast(yellow line–fast track 

exclusion zone and blue line–revised exclusion zone). 

Figure B-21. Shipborne Radar 5–Exclusion zone middle west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
67

Figure B-22. Shipborne Radar 5–Exclusion zone lower west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-23. Shipborne Radar 5–Exclusion zone upper east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
68

Figure B-24. Shipborne Radar 5–Exclusion zone middle east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-25. Shipborne Radar 5–Exclusion zone lower east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
69

Figure B-26. Shipborne Radar 5–Exclusion zone eastern gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-27. Shipborne Radar 5–Exclusion zone western gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

The composite exclusion zones for the shipborne radars operating in the 3550–3650 MHz band 

are provided in Figures B-28 through B-36. The yellow line on each plot is the composite 
70

exclusion zone distance based on the Fast Track Report and the blue line is the revised composite 

exclusion zone distance. Ship locations are not shown in these figures.

Figure B-28. Composite shipborne radar exclusion zone lower 48 states (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-29. Composite shipborne radar exclusion zone upper west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone). 
71

Figure B-30. Composite shipborne radar exclusion zone middle west coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone).

Figure B-31. Composite shipborne radar exclusion zone lower west coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
72

Figure B-32. Composite shipborne radar exclusion zone upper east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-33. Composite shipborne radar exclusion zone middle east coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
73

Figure B-34. Composite shipborne radar exclusion zone lower east coast(yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).

Figure B-35. Composite shipborne radar exclusion zone eastern gulf coast (yellow line–fast track 

exclusion zone and blue line–revised exclusion zone).
74

Figure B-36. Composite shipborne radar exclusion zone western gulf coast (yellow line–fast 

track exclusion zone and blue line–revised exclusion zone).

The results for the shipborne radar exclusion zone distance analysis compared to the Fast Track 

Report exclusion zone distances are provided in Tables B-1 to B-3. The results are presented for 

three parameters: percentage of area impacted, percentage of population impacted, and average 

exclusion zone distance. 

Table B-1. Summary of Shipborne Radar Analysis Results

Radar Identifier Geographic Area

Percentage of Area Impacted

(km2)

Fast Track13 Report 

Analysis

Revised Analysis

(3.5 GHz Study)

Reduction of Area 

Impacted

Shipborne Radar 1

East Coast

West Coast

Gulf Coast

654,067

477,627

528,374

251,706

76,363

167,936

62 %

84 %

68 %

Shipborne Radar 4

East Coast

West Coast

Gulf Coast

1,081,389

611,070

880,096

299,243

79,729

182,258

72 %

87 %

79 %

Shipborne Radar 5

East Coast

West Coast

Gulf Coast

1,040,991

956,435

718,117

339,037

93,530

211,218

67 %

90 %

71 %

 13 [B-1] Table 5-4.
75

Table B-2. Comparison of Summary of Shipborne Radar Analysis Results

Radar Identifier

Geographic 

Area

Fast Track Report 

Analysis

Revised Analysis

(3.5 GHz Study) Reduction in 

Percentage 

of Total 

Population 

Impacted

Population 

Impacted

Percentage 

of Total 

Population 

Impacted

Population 

Impacted

Percentage 

of Total 

Population 

Impacted

Shipborne Radar 1

East Coast

West Coast

Gulf Coast

58,562,753

41,164,529

18,943,514

28 %

20 %

9 %

48,387,330

27,112,935

12,439,455

23 %

13 %

6 %

17 %

34 %

34 %

Shipborne Radar 4

East Coast

West Coast

Gulf Coast

66,771,384

42,136,232

29,342,307

32 %

20 %

14 %

49,480,866

27,092,000

12,812,822

24 %

13 %

6 %

26 %

36 %

56 %

Shipborne Radar 5

East Coast

West Coast

Gulf Coast

66,589,823

45,145,947

21,992,519

32 %

22 %

11 %

50,391,618

28,447,839

12,878,464

24 %

14 %

6 %

24 %

37 %

41 %

Table B-3. Summary of Shipborne Radar Analysis Results by Area Affected

Radar Identifier

Geographic 

Area

Average Exclusion Zone Distance

Fast Track Report 

Analysis

Revised Analysis

(3.5 GHz Study)

Reduction in Average 

Exclusion Zone 

Distance

Shipborne Radar 1

East Coast

West Coast

Gulf Coast

255

189

248

89

41

64

166

148

184

Shipborne Radar 4

East Coast

West Coast

Gulf Coast

426

245

427

117

39

71

309

206

356

Shipborne Radar 5

East Coast

West Coast

Gulf Coast

398

423

359

116

46

94

282

377

265

B.3 References

[B-1] U.S. Department of Commerce, National Telecommunications and Information 

Administration, An Assessment of the Near-Term Viability of Accommodating Wireless 

Broadband Systems in the 1675-1710 MHz, 1755-1780 MHz, 3500-3650 MHz, and 4200-

4220 MHz,4380-4400 MHz Bands (Nov. 15, 2010) (Fast Track Report), available at 

http://www.ntia.doc.gov/reports/2010/FastTrackEvaluation_11152010.pdf.
76

APPENDIX C GROUND-BASED RADAR ANALYSIS RESULTS

C.1 Introduction

Appendix C provides the results of the exclusion zone distance analysis and the exclusion zones 

needed to protect the ground-based radars operating adjacent to the 3550–3650 MHz band. 

C.2 Ground-Based Radar Analysis Results

Exclusion zones around ground-based radars are defined as a fixed distance outside the 

perimeters of the installations listed in Tables C-1 and C-2.

Table C-1. Ground-Based Radar 1 Locations 

Location of Military Facility 

Aberdeen PG, MD

Camp Shelby, MS

Camp Sherman, OH

Camp Smith, NY

Fort Benning, GA

Fort Bragg, NC

Fort Campbell, KY

Fort Carson, CO

Fort Drum, NY

Fort Indiantown Gap, PA

Fort Irwin, CA

Fort Hood, TX

Fort Lewis, WA

Fort Polk, LA

Fort Riley, KS

Fort Sill, OK

Fort Stewart, GA

Letterkenny, PA

Morristown, NJ

Pohakuola Training Area, HI

Yakima Firing Center, WA

Yuma PG, AZ

Schofield Barracks, HI

Knoxville, TN

Pinon Canyon, CO

Tupelo, MS

White Sands Missile Range, NM
77

Table C-2. Ground-Based Radar 3 Locations

Location of Military Facility

MCB Camp Pendleton, CA

MCAS Miramar, CA

MAGTFTC Twenty Nine Palms, CA

MCAS Yuma, AZ

MCB Camp Lejeune, NC

MCAS Cherry Point, NC

MCAS Beaufort, SC

Virginia Beach, VA (reserve unit)

Fort Worth, TX (reserve unit)

Fort Sill, OK

Naval Air Station, Patuxent River, MD

Eglin AFB, FL

Naval Air Station China Lake, CA

Table C-3. Results of Ground-Based Radar Analyses

Military Base Radar Nearby City Exclusion Zone

Fort Carson CO GB-1 Colorado Springs CO 2 km

Fort Riley KS GB-1 Junction City KS 3 km

Fort Riley KS GB-1 Manhattan KS 2 km

Fort Sill OK GB-3 Lawton OK 2 km

Camp Pendleton CA GB-3 Oceanside/Carlsbad CA 4 km

Based on these results, a 3 kilometer exclusion zone around the perimeter of each of the 

installations listed in Tables C-1 and C-2 is needed to protect these radars. The exclusion zones 

for ground-based radars in the Fast Track Report were between 40 and 60 km.14

The exclusion zones for the ground-based radar systems are provided in Figures C-1 through 

C-8.

 14 [C-1] Table 5-2.
78

Figure C-1. Ground-based radar exclusion zone lower 48 states.

Figure C-2. Ground-based radar exclusion zone upper west coast.
79

Figure C-3. Ground-based radar exclusion zone middle west coast.

Figure C-4. Ground-based radar exclusion zone upper central.
80

Figure C-5. Ground-based radar exclusion zone lower central.

Figure C-6. Ground-based radar exclusion zone upper east coast.
81

Figure C-7. Ground-based radar exclusion zone middle east coast.

Figure C-8. Ground-based radar exclusion zone lower east coast.
82

C.3 References

[C-1] U.S. Department of Commerce, National Telecommunications and Information 

Administration, An Assessment of the Near-Term Viability of Accommodating Wireless 

Broadband Systems in the 1675-1710 MHz, 1755-1780 MHz, 3500-3650 MHz, and 4200-

4220 MHz,4380-4400 MHz Bands (Nov. 15, 2010) (Fast Track Report), available at 

http://www.ntia.doc.gov/reports/2010/FastTrackEvaluation_11152010.pdf.
NTIA FORM 29 U.S. DEPARTMENT OF COMMERCE 

(4-80) NATIONAL TELECOMMUNICATIONS AND INFORMATION ADMINISTRATION

BIBLIOGRAPHIC DATA SHEET

1. PUBLICATION NO.

TR-15-517

2. Government Accession No. 3. Recipient’s Accession No.

4. TITLE AND SUBTITLE

3.5 GHz Exclusion Zone Analyses and Methodology

5. Publication Date

June 2015, Reissued March 2016

6. Performing Organization Code

NTIA/OSM

7. AUTHOR(S)

E. Drocella, J. Richards, R. Sole, F. Najmy, A. Lundy, and P. McKenna

9. Project/Task/Work Unit No.

8. PERFORMING ORGANIZATION NAME AND ADDRESS 6469000-200

Institute for Telecommunication Sciences

National Telecommunications & Information Administration

U.S. Department of Commerce

325 Broadway

Boulder, CO 80305

10. Contract/Grant Number.

11. Sponsoring Organization Name and Address

National Telecommunications & Information Administration

Herbert C. Hoover Building

14th & Constitution Ave., NW

Washington, DC 20230

12. Type of Report and Period Covered

14. SUPPLEMENTARY NOTES

15. ABSTRACT (A 200-word or less factual summary of most significant information. If document includes a significant bibliography or

literature survey, mention it here.)

This report describes the 3.5 GHz Study. It explains the assumptions, methods, analyses, and system characteristics used to 

generate the revised exclusion zones for small-cell commercial broadband systems to protect federal radar operations (ship 

and land based) from aggregate interference in the band 3550–3650 MHz. The 3.5 GHz Study’s exclusion zones are 

compared with the exclusion zones that were generated in the Fast Track Report, which considered macro-cell operations.

16. Key Words (Alphabetical order, separated by semicolons)

NTIA, FCC, radar, exclusion zones, CBRS, CBSD, spectrum, sharing, 3550–3650 MHz 

17. AVAILABILITY STATEMENT

UNLIMITED.

FOR OFFICIAL DISTRIBUTION.

18. Security Class. (This report)

Unclassified

20. Number of pages

103

19. Security Class. (This page)

Unclassified

21. Price:

N/A

NTIA FORMAL PUBLICATION SERIES

NTIA MONOGRAPH (MG) 

A scholarly, professionally oriented publication dealing with state-of-the-art research or 

an authoritative treatment of a broad area. Expected to have long-lasting value.

NTIA SPECIAL PUBLICATION (SP)

Conference proceedings, bibliographies, selected speeches, course and instructional 

materials, directories, and major studies mandated by Congress.

NTIA REPORT (TR)

Important contributions to existing knowledge of less breadth than a monograph, such as 

results of completed projects and major activities.

JOINT NTIA/OTHER-AGENCY REPORT (JR)

This report receives both local NTIA and other agency review. Both agencies’ logos and 

report series numbering appear on the cover. 

NTIA SOFTWARE & DATA PRODUCTS (SD)

Software such as programs, test data, and sound/video files. This series can be used to

transfer technology to U.S. industry.

NTIA HANDBOOK (HB)

Information pertaining to technical procedures, reference and data guides, and formal 

user's manuals that are expected to be pertinent for a long time.

NTIA TECHNICAL MEMORANDUM (TM)

Technical information typically of less breadth than an NTIA Report. The series includes 

data, preliminary project results, and information for a specific, limited audience.

For information about NTIA publications, contact the NTIA/ITS Technical Publications Office at 

325 Broadway, Boulder, CO, 80305 Tel. (303) 497-3572 or e-mail info@its.bldrdoc.gov.