**Arizona Absolute Ground Control Inspection on the MDOQ Imagery**

**Zack Adkins**

**APFO Service Center Support Section**

**Background**

In 2007, imagery for the state of Arizona was acquired with full state,
1 meter NAIP coverage. In previous years, NAIP imagery had been tied to
a relative accuracy specification; for Arizona NAIP 2007, this
specification was changed to absolute accuracy. This meant the imagery
would be tied to true ground control points instead of older imagery
datasets. Once the true ground control was acquired from different
sources, the points were then implemented into the NAIP inspection
process. In the NAIP inspection process, the ground control points were
used. However the points had never been checked against the older
imagery dataset (MDOQs (mosaicked digital ortho quarter quads) loaded on
the APFO GDW (geodata warehouse)). This document will outline that
process, focusing on the results, statistics, and the comparison of the
MDOQ inspection results to the NAIP inspection results.

**Inspection Methodology**

The inspection of the ground control points on the MDOQ imagery used the
same process as the NAIP inspection. First, a control point shapefile
was loaded into ArcMap. Then, a new empty shapefile was created for the
MDOQ control point data. Next, imagery was loaded from the APFO GDW.
Inspection began once all layers were loaded. Based upon supplemental
data (see figure 1), the photo identifiable control points were located
and digitized on the MDOQ imagery where the inspector deemed valid (see
figure 1). After digitizing, each point was then assigned a "quality"
value from 1 to 5 (1 being the best, 5 meaning a recommendation of
removal from the inspection database). Once all control points for the
state were checked, then the "Point Distance" tool was used to check the
offset distance between the true ground points and the points digitized
on the MDOQ. This data enabled the inspector to determine the horizontal
accuracy of the imagery as well as the reliability of the true ground
control point dataset.

![](media/image1.png){width="7.752083333333333in"
height="4.845138888888889in"}

*Figure 1: Digitizing control on the MDOQ imagery (the red dot is the
digitized point).*

**MDOQ Inspection Statistics**

For Arizona NAIP 2007, there were 544 true ground control points. Of the
544, 527 were used in the MDOQ point inspection process. The reasons for
not inspecting 17 of the points were various; usually the feature picked
did not exist on the older MDOQ imagery. The overall point offset
statistics for the 527 points that were checked is shown below:

**Minimum Offset Distance**: 0.105758 meters

**Maximum Offset Distance**: 19.809101 meters

**Mean Offset Distance**: 2.59158 meters

**Standard Deviation**: 1.951404 meters

**RMSE**: 3.244112 meters

The overall percent accuracy for the entire state was **94.877%.** Of
the 527 points, 27 were over 6 meters offset from true ground, which is
outside (barely) the 2007 NAIP absolute control specification. A
histogram illustrating the point offset is illustrated in figure 2
below. Blue represents points within tolerance; red represents points
out of tolerance.

![](media/image2.wmf){width="7.995833333333334in"
height="5.455555555555556in"}

*Figure 2: Histogram showing offset with point frequency*

The point quality rating is a good way of judging each point's overall
value as a photo identifiable, true ground control point. Each point
that was digitized received a value of 1 through 5. Points rated a "1"
are of the highest quality; "5" ratings are recommended to be deleted
altogether. Here are the statistics for the point quality ratings:

> **"1" Rating**: 3 points
>
> **"2" Rating:** 128 points
>
> **"3" Rating:** 322 points
>
> **"4" Rating:** 59 points
>
> **"5" Rating:** 15 points

From this data, it can be observed that most of the points were of
average quality, but were still very usable in the inspection.

![](media/image3.wmf){width="6.497222222222222in"
height="4.615277777777778in"}

*Figure 3: Graph showing point quality ratings percentages*

Here is the breakdown of the point quality ratings within tolerance (6
meters offset from true ground):

> **1 Rating**: 100% within tolerance
>
> **2 Rating**: 97% within tolerance
>
> **3 Rating**: 94% within tolerance
>
> **4 Rating**: 97% within tolerance
>
> **5 Rating**: 87% within tolerance

There is nothing out of the ordinary here; the better the rating, the
more likely the point is to be within tolerance and vice versa.

**Comparing MDOQ Inspection Statistics with NAIP Inspection Statistics**

The MDOQ points were inspected exactly the same way as the NAIP points.
The statistical methods and collections were identical; however the
statistical outcomes will be different. In the NAIP inspection there
were 544 points to be measured; this time 530 points were inspected with
14 points not inspected. In this case, many of the ground control
features no longer existed on the NAIP imagery. Here are the general
offset statistics between the true ground control points and the 2007
NAIP imagery:

**Minimum Offset Distance**: 0.058385 meters

**Maximum Offset Distance**: 20.592761 meters

**Mean Offset Distance**: 2.114408 meters

**Standard Deviation**: 1.946102 meters

**RMSE**: 2.873679 meters

The graph below illustrates how close the point offset statistics are
between points digitized on the NAIP imagery and the MDOQ imagery.

![](media/image4.wmf){width="7.627083333333333in"
height="4.589583333333334in"}

*Figure 4: Graph showing NAIP and MDOQ offset statistics*

The quality ratings for the control points differed somewhat between the
NAIP imagery and the MDOQ imagery.

![](media/image5.wmf){width="7.876388888888889in"
height="4.486805555555556in"}

*Figure 5: Graph showing NAIP and MDOQ point quality ratings*

The graph above illustrates that the points on the NAIP imagery had five
times as many "1" ratings. This is due to the fact that the NAIP imagery
is generally of a higher quality, newer, and the state of Arizona
acquired new control thus making the better points more photo
identifiable. The MDOQ imagery had more points rated as average (3) and
half as many recommended for inspection database removal (5). Most of
the points rated a "5" on the NAIP imagery were selected on features
that have changed over the last 10 years or on features that no longer
exist.

It would also be worthwhile to compare the point statistics based upon
accuracy order. The accuracy order for the control points ranged from
sub foot to two meter. In this comparison, the highest order accuracy
will be examined (sub foot). There were 119 points that were sub foot
accuracy. These are the offset statistics for the sub foot points
measured on the NAIP imagery:

> **Minimum Offset Distance**: 0.210594 meters

**Maximum Offset Distance**: 4.665197 meters

**Mean Offset Distance**: 1.575352 meters

**Standard Deviation**: 0.9974 meters

**RMSE**: 1.865109 meters

These are the offset statistics for the sub foot points measured on the
MDOQ imagery:

> **Minimum Offset Distance**: 0.105758 meters

**Maximum Offset Distance**: 13.508835 meters

**Mean Offset Distance**: 2.764579 meters

**Standard Deviation**: 2.24229 meters

**RMSE**: 3.559877 meters

The following histograms (figures 6 and 7) illustrate the point offsets
for the NAIP and MDOQ points. None of the sub foot points measured on
the NAIP imagery was out of tolerance; 7 of the sub foot MDOQ points
were out of tolerance.

![](media/image6.wmf){width="7.003472222222222in"
height="4.771527777777778in"}

*Figure 6: Histogram showing point offset frequency for NAIP*

![](media/image7.wmf){width="7.499305555555556in"
height="5.110416666666667in"}

*Figure 7: Histogram showing point offset frequency for MDOQ imagery*

The graph in figure 8 illustrates the difference between the sub foot
points on the NAIP imagery and the MDOQs. With the higher order accuracy
points, the differences between the NAIP points and the MDOQ points is
much greater than the point dataset as a whole. The RMSE was twice as
high for the MDOQs than the NAIP. This emphasizes that a highly
accurate, photo identifiable point located on high quality imagery can
be very valuable.

![](media/image8.wmf){width="7.752083333333333in"
height="4.979166666666667in"}

*Figure 8: Graph comparing MDOQ and NAIP offsets on sub foot accuracy*

The comparison of the distance between MDOQ points and the NAIP points
shows some interesting outcomes. Of the 544 total control points, 515
were digitized on both the MDOQ and the NAIP imagery. The following are
the offset statistics between the MDOQ points and the NAIP points
measured at each common absolute control station:

> **Minimum Offset Distance**: 0.094528 meters

**Maximum Offset Distance**: 12.342549 meters

**Mean Offset Distance**: 2.865047 meters

**Standard Deviation**: 1.77895 meters

**RMSE**: 3.37241 meters

The two following examples illustrate the offset between the MDOQ points
and the NAIP points in relation to the absolute ground control. The
yellow point is the true ground control point, green is the inspection
point digitized on NAIP, and red is the inspection point digitized on
the MDOQ. The distance between the NAIP and MDOQ points is 12.34 meters.
This was the largest offset distance value. Notice how the point
digitized on the NAIP image is more accurate. It is only 1.18 meters
from true ground; the MDOQ point is 13.51 meters from true ground.

![](media/image9.jpeg){width="7.254166666666666in"
height="4.940277777777778in"}

*Figure 9: Inspection point offset on NAIP image*

![](media/image10.jpeg){width="7.254166666666666in"
height="4.940277777777778in"}

*Figure 10: Inspection point offset on MDOQ image (the yellow point is
beneath the green point)*

In figure 11, the frequency distribution of the offset values between
the MDOQ and NAIP points is illustrated. Most of the values are between
2.5-3.5 meters.

![](media/image11.wmf){width="7.251388888888889in"
height="4.940972222222222in"}

*Figure 11: Histogram of point distance between NAIP and MDOQ points*

Although there is no distance specification tolerance set for offset
between the MDOQ and the NAIP points, one could hypothetically create a
tolerance. Currently, the tolerance for absolute accuracy control is 6
meters offset from true ground. If you imagine a right isosceles
triangle (see figure 12), then a distance tolerance between the NAIP and
MDOQ points could be created. Using the Pythagorean Theorem (a^2^ + b^2^
= c^2^), that distance would be 8.5 meters. If this were the
specification, 6 of the 515 point distances would be out of tolerance,
leading to **99%** accuracy between the MDOQ and NAIP imagery.

*Figure 12: Estimating offset tolerance between MDOQ and NAIP points*

**Conclusions**

These previous data examples are just a few that one can do for a
statistical review. Many different conclusions can be drawn from these
statistics. The graph in figure 4 illustrates the overall closeness
between the MDOQ point offsets from true ground and the NAIP point
offsets from true ground; none of the overall statistical categories
have a large amount of variance. The overall percentage horizontal
accuracies (for AZ) were close also: **94.877%** for the points on the
MDOQs and **95.660%** for the points on the NAIP imagery. Perhaps the
greatest deduction can come from the offset distance statistics between
the NAIP points and the MDOQ points. Although there is no specification
in the NAIP contract regarding this, there were only 6 distances out of
515 that were out of the hypothetical tolerance (8.5 meters). As it was
stated above, this is **99%** within tolerance. Could it then be said
that the two point data sets (MDOQ and NAIP) for the entire state of
Arizona are 99% accurate to each other?

One may ask, "Why move to an absolute accuracy specification if the
offset results are so close?" The relative accuracy specification
(relative to the MDOQs) has some issues. NAIP imagery horizontally tied
to the MDOQs is potentially less accurate as an image base layer. It can
be less accurate to digitize upon (e.g. CLU polygons), less accurate for
other vector datasets in a GIS, and less valuable as a product. This was
illustrated in the comparison of the higher order accuracy control
points. The absolute ground control accuracy specification is more
advantageous in the long run. The specification is tighter and more
understandable than the relative specification. Image datasets may not
become more accurate immediately, but will over time. Since the NAIP
imagery is used in a GIS, a more accurate dataset is a more valuable
dataset. With the move to absolute ground control as a horizontal
accuracy specification, there will be less manipulation of datasets over
time to correct errors. Since this control point evaluation was only
done on one state, it would be viable to perform this analysis on
another state in the future.

![](media/image12.jpeg){width="5.747916666666667in"
height="8.621527777777779in"}

*Map 1: Control point offset between true ground and the MDOQs*

![](media/image13.jpeg){width="5.831944444444445in"
height="8.747916666666667in"}

*Map 2: Control point quality ratings on MDOQ imagery*

![](media/image14.jpeg){width="5.831944444444445in"
height="8.747916666666667in"}

*Map 3: Control point quality ratings with offset distance*