**Peer Review Panel Report**

**On**

**OET Report FCC/OET 07-TR-1006**

**Evaluation of the Performance of Prototype TV-Band White Space
Devices**

Peer Reviewers:

George Dillon, EB

James Higgins, EB/Columbia

Martin Liebman, WTB

Doug Miller, EB/Atlanta

Mary Shultz, WTB

**Background:**

This report contains the peer review of OET Report FCC/OET 07-TR-1006,
entitled "Evaluation of the performance of Prototype TV-Band White
Spaces Devices." The project was undertaken in support of the Federal
Communications Commission's (FCC) ongoing proceeding to consider rules
for permitting low power radio transmitting devices to operate on an
unlicensed basis in the frequency bands that are currently allocated to
the Television Broadcast and certain other licensed services. As part of
this proceeding, OET has conducted a study of two prototype devices with
capabilities for operating on an unlicensed basis in the TV bands. The
two prototype devices, submitted by industry in response to an OET
public notice inviting submittal of prototype white space devices, are
actually product development platforms rather than models of products
that could be marketed. One of these devices has both transmitting and
spectrum sensing capabilities; the other has only spectrum sensing
capability.

Peer review of the report was performed as required under the OMD
Information Improvement Act for influential scientific and engineering
studies. The review panel was made up of five engineers, three from the
Enforcement Bureau and two from the Wireless Telecommunications Bureau.
The review panel analyzed and discussed various subject areas in the OET
report, both independently and jointly. Specifically, as requested in
the OET memo, the review panel addressed the following:

1.  Whether the scope of testing in terms of spectrum sensing abilities
    and signal conditions examined was appropriate and sufficient;

2.  Whether the measurement methodologies used in the testing of the
    prototype devices spectrum sensing abilities was appropriate;

3.  Whether the scope of testing of the first prototype (the device with
    transmitting capability) for its potential to cause interference to
    digital TV, analog TV, and wireless microphone signals was
    appropriate

4.  Whether the various tests performed were properly conducted
    consistent with the selected methodologies.

The response of the review panel is presented below for each question
shown above:

**1) Whether the scope of testing in terms of spectrum sensing abilities
and signal conditions examined was appropriate and sufficient.**

In the opinion of the review panel, the overall scope of the spectrum
sensing testing was appropriate.

For all tests, measurements were taken across the entire operating range
of the prototype devices which consisted of UHF channels 21-51. With
respect to television services, bench tests were performed for both the
Prototype A devices and the Prototype B device. The TV portion of the
sensor testing examined the ability of the prototype devices to detect
the signals of both analog and digital full service stations. In
addition, field tests were performed for the Prototype A devices. Field
tests were performed utilizing live over-the-air ("OTA") DTV and NTSC
television signals. Tests were taken at 4 different sites where OTA
television broadcasts were being received. Field tests were not
performed for the Prototype B device because the supplier formally
declared that the device was not suitable for field testing and
requested that it not be included in those tests. The wireless
microphone portion of the testing was performed in the laboratory only,
using signals generated by a wireless microphone. The review panel
believes that the approach taken and procedures followed by the staff
were well thought out and provided an excellent analysis of the sensing
capabilities of the WSDs.

The review panel notes that Tables 3-8 and 3-9 contain graphs of the
two-channel detection threshold test results for WSD Prototype A and WSD
Prototype B respectively. Table 3-8 appears to indicate that the
addition of N-1 and N+2 transmissions at the -60 dBm signal level, which
the report characterizes as a "'strong' signal level relative to the
amplitude in the detection channel," did not have any effect on the
sensing capabilities of the Prototype A, but adjacent channel signals at
that level did appear to have an effect on Prototype B (as shown in
Table 3-9). The review panel felt that it would have been useful to have
tested stronger N-1 and N+2 signals as well because higher level
adjacent channel OTA signals can exist at a DTV receiver (see, e.g.,
Table 5-1). Higher level adjacent channel signals might have affected
Prototype B's ability to sense a co-channel signal more than the
moderate-level adjacent channel signal tested and the panel believes
that it would have been interesting to have examined this condition. In
addition, the review panel believes that if more time had been available
to conduct the test, it would be more realistic to have tested multiple,
adjacent DTV signals at the same time, rather than testing only a single
adjacent channel at a time.

In addition, the report indicates that "*No attempt was made to quantify
the received TV signal levels at each measurement location as such data
was not necessary for this assessment of the performance of the sensing
function. Instead, map-based plots showing the service contour
associated with each licensed full service and low power TV broadcast
station within a 150-km (94 miles) radius of the test site coordinates
was generated for each test site*." Due to the limited time available to
conduct the test, the review panel agrees that it may not have been
feasible to take actual measurements of received TV signals. But the
review panel feels that since the success of the WSD in sensing TV
signals is dependent on the strength of those signals, it would have
been useful to have taken some sample measurements to perhaps quantify
one or more "high," "medium," and "low-signal\" stations, to try to
examine the correlation between TV signal level and sensing capability
in the field, and to see how those results compared with the results
obtained in the bench tests. If, for example, information was known at
Site 1 about the signal levels of Channels 21, 23, 25, 30, and 31, where
the WSD made perfect decisions (see Table 3-2), versus the signal levels
of Channels 43, 44, 47, and 49, where a number of incorrect decisions
were made, then possible correlations between signal level and WSD
sensing capabilities could have been made. Similarly, at Site 2, where
the WSD no longer performed perfectly at Channels 21, 23, and 25 (see
Table 3-5), information about the signal level on those channels, as
well as information about the signal levels of N-1, N-2, N+1, and N+2
channels, might have enabled an analysis of the correlation between
signal level of the channels being tested, the signal levels of first
and second adjacent channels, and the WSD's sensing capabilities.

**2)** **Whether the measurement methodologies used in the testing of
the prototype devices spectrum sensing abilities was appropriate.**

The review panel believes that the measurement methodologies used in the
testing of the prototype devices were appropriate. Guidance and
procedures established to date by IEEE 802.22 for testing the spectrum
sensing capability of fixed/access WSDs were considered where
applicable. In the instances where the testing deviated from those
methodologies, the review panel feels that the underlying objectives
were met by the tests that were performed.

As stated above, both bench and field tests were performed to assess the
scanning/sensing capability of the prototype WSDs for the television
portion of the testing. Two separate bench tests were performed to
determine the minimum DTV signal detection threshold for each of the two
prototype devices delivered to the laboratory. The first bench test
utilized a single, unimpaired, laboratory-grade DTV signal as the test
input. The second bench test utilized two unimpaired, laboratory-grade
signals as the input, one on the detection channel and the other placed
on one of two adjacent channels and held at a constant amplitude.
Measurements were taken to determine 1) the minimum discernable signal
that could successfully be detected by the scanner/sensor component of
the prototype and 2) the associated reliability of detection.

Multiple measurements were taken in order to determine the percentage of
successful detections with some statistical relevance, and initial tests
were performed for multiple channels to investigate potential frequency
related differences in performance. For the baseline detection signal
test using a single DTV input, the input DTV signal was initially set to
a low, but measurable, level and then further attenuated incrementally
with the calibrated step attenuator bank while exercising the scanner
over the occupied channel. At each attenuation step (input power level),
thirty independent trials were performed. Tests were performed on three
channels in the lower (channel 21), middle (channel 36) and upper
(channel 51) portions of the WSD tuning range. Considering the time
allowed for this study, the review panel feels that the number of trials
and channels tested was sufficient.

Since the initial test results revealed that the sensing performance of
the devices was fairly consistent over their tuning range, the remaining
tests were performed on a single channel in the middle of the tuning
range. The review panel agrees that testing on a single channel was
appropriate given the results of the baseline testing.

The methodology for the two-DTV input signal test was similar to the
methodology used in the single DTV input signal tests but with the
addition of a second DTV input signal. At each attenuation step (power
level in detection channel), thirty independent trials were performed in
order to determine the percentage of successful detections with some
statistical relevance. The second DTV signal was introduced at N-1 for
one test and at N+2 for another test and held at a constant level.
Again, at each attenuation step (power level in detection channel),
thirty independent trials were performed in order to determine the
percentage of successful detections with some statistical relevance.

Field tests were performed utilizing live OTA DTV and NTSC television
signals. Testing was conducted at several independent locations at each
field test site. At each of these independent locations, the prototype
was used to scan over its entire channel space and the results were
recorded. This process was repeated to produce results from three
independent trials at each test location. The review panel felt that the
measurements taken at these locations provide a good initial assessment
of the scanners/sensors performance under "real world" conditions.

The review panel also feels that the measurement methodologies used for
the wireless microphone portion of the testing were appropriate to
ascertain the ability of the Prototype A and Prototype B sensors to scan
for and detect Part 74 wireless microphones. Bench tests of the
Prototype A and Prototype B devices ability to sense wireless
microphones were performed using signals generated by wireless
microphones. These signals were coupled directly to the input terminals
of the prototype devices. Three different Part 74 wireless microphone
systems were used in these tests, but digitally modulated wireless
microphones were not tested. Regarding these tests, the review panel did
note that a continuously modulated signal was used to modulate the
wireless microphones while the scanning was being performed. Typical
analog wireless microphone use will often be a continuous carrier
intermittently modulated (such as, for example, use during news reports,
program production, speeches, etc.). Therefore it should be pointed out
that the continuously modulated signals used in the testing likely
represent \"best case\" detection opportunities. Given more time, the
review panel felt that an additional test could be done using an
unmodulated carrier or intermittently modulated carrier to represent
\"worst case\" detection opportunity.

**3) Whether the scope of testing of the first prototype (the device
with transmitting capability) for its potential to cause interference to
digital TV, analog TV, and wireless microphone signals was
appropriate.**

It is the opinion of the review panel that the scope of the testing of
the first prototype device for its potential to cause interference to
digital TV and wireless microphones was appropriate, given the study's
stated constraints. However, the review panel did not observe any
discussion in the report describing tests that may have been done to
analyze potential interference to analog TV signals.

The project conducted limited, or "anecdotal," tests in the field of the
prototype WSD transmitter to provide information on its potential to
interfere with TV reception. These tests were performed in a large
outdoor area to evaluate the worst-case performance with an unobstructed
line-of-sight (LOS) propagation path between the WSD transmit antenna
and the DTV test receiver antenna. A test DTV receiver was placed in the
area and connected to an indoor antenna with the antenna oriented
towards a DTV transmitter on channel 29. The WSD transmitter was then
placed in the "main beam" of the receive antenna, tuned to the same
channel, and activated at incremental distances from the DTV receive
antenna while observing for interference effects to the picture quality.
Tests were also performed with the WSD tuned to adjacent channels both
with and without the use of the external transmit filter. The distance
at which interference was observed was measured and recorded. Overall,
the review panel feels that these tests provided an excellent analysis
of the effects of WSDs on DTV reception.

The report indicates that it was necessary to employ over-the-air
Channel 29 as the desired signal, even though that channel produced a
signal, at -63.5 dBm, that was some 20 dB higher than the threshold of
visibility (TOV). The panel recognizes that use of OTA signals produces
only anecdotal results and concurs with the report\'s observations on
\"OTA Interference Tests\" in Section 6.3, but the panel believes that
it would have been useful for a few additional tests to have been
conducted to try to analyze interference to TV reception under various,
additional conditions.

For example, in the test the WSD was placed in the main beam of the DTV
antenna in the direction of the DTV station, which maximized the WSD\'s
potential to cause interference to DTV reception. The panel believes
that it would have been useful to have placed the WSD "off-beam" at two
or three different angles and observed the results. Similarly, because
the DTV antenna was pointed directly at the DTV station to achieve
maximum signal, this produced a best-case scenario for the WSD in terms
of its ability to cause interference to DTV reception. So perhaps a test
where the DTV antenna did not point directly at the DTV station could
have been done so that an examination of the effect of the WSD on DTV
reception could have been made under a weaker DTV signal condition.

The tests also looked at the susceptibility of wireless microphones to
the signals emitted by the Prototype A transmitter and also simulated
WSD signals generated by using broadband signals modulated using several
alternative methods. Wireless microphone testing was conducted in the
laboratory using both these simulated WSD signals and signals from the
Prototype A transmitter. Three different Part 74 wireless microphone
systems were used in these tests. The simulated WSD signals consisted of
an audio modulated FM signal, a wideband noise signal and a wideband
OFDM signal. In these tests, interference was defined to occur at the
point where the signal-to-noise plus distortion (SINAD) ratio reading at
the audio output of the microphone receiver was 30 dB.

In general, the review panel feels that, the scope of the wireless
microphone interference testing was appropriate to provide a basic
understanding of susceptibility of wireless microphones to interference.
The review panel feels that a few over-the-air tests conducted with the
wireless microphones would have helped to verify the direct-coupled
results.

**4) Whether the various tests performed were properly conducted
consistent with the selected methodologies.**

In the opinion of the review panel, the tests were properly conducted
consistent with the selected methodologies. The review panel believes
that the testing was well done and thorough. For more description, see
No 2 above.

The review panel notes that the information contained in Table 3-13,
Summary of Field Test Data with Prototype A Version 2, for \"Site 2\"
NTSC observations is inconsistent with what is described in the text.
The table indicates that a NTSC signal was viewable on the TV for Site
2, but the text indicates that the converter box at Site 2 did not
include an NTSC tuner and thus it was not possible to verify whether an
analog TV signal could be viewed.

Also, in the OTA interference test (Section 5.1), the report describes
that \"receiver #I1\" from a previous study was selected to be used in
the test. The review panel questions the basis for the selection. The
review panel believes that it would be useful to know if receiver #I1
selected randomly; if it was selected because it was in the median range
of performance in the previous study; or if it was selected for another
reason.