Department of the Interior

US Geological Survey

USGS QUality Assurance Plan\
for Digital Aerial Imagery:

Part 2:\
Type Certification of Sensor System

DRAFT Version 0.7

February 2008

![](media/image1.png)

Digital Aerial Sensor Type Certification

(Draft Version 0.7)

February 2008

Reviewed By:

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

George Y.G. Lee Date

Digital Raster Theme Coordinator

US Geological Survey

Approved By:

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Greg Stensaas Date

Project Chief

Remote Sensing Technologies Project

US Geological Survey, EROS Center

USGS EROS Data Center

Sioux Falls, South Dakota

# Executive Summary {#executive-summary .Preface-Heading}

This document describes the process and areas of interest for the USGS
Digital Aerial Sensor Type Certification. This certification is one part
of the four-part *USGS Quality Assurance Plan for Digital Aerial
Imagery* (the "Plan"). This Plan is described in an overview document
\[insert ref. XXXX\] and consists of four parts:

> 1\) Federal Digital Imagery General Contracting Guidelines
>
> 2\) Digital Aerial Sensor Type Certification
>
> 3\) Data Provider Certification
>
> 4\) Data Acceptance Standards and Guidelines

The other portions of the Plan are described in other documents.

*Keywords: digital, aerial, manufacturer, quality assurance, sensor.*

# Contents {#contents .Preface-Heading}

[Executive Summary iii](#executive-summary)

[Contents iv](#contents)

[1 The USGS Digital Aerial Sensor Certification
1](#the-usgs-digital-aerial-sensor-certification)

[1.1 Background and Introduction 1](#background-and-introduction)

[1.2 Procedure for Obtaining Type Certification
1](#procedure-for-obtaining-type-certification)

[2 Information to be Provided by the Manufacturer
2](#information-to-be-provided-by-the-manufacturer)

[Appendix A: Required Information 3](#__RefHeading___Toc191276622)

[1 Technical Questions 3](#technical-questions)

[1.1 Basic System Design 3](#basic-system-design)

[1.2 Optics 3](#optics)

[1.3 Geometry/Geodesy 3](#geometrygeodesy)

[1.4 Radiometry 4](#radiometry)

[1.5 Spatial Performance 6](#spatial-performance)

[2 System Calibration 6](#system-calibration)

[2.1 Factory Geometric Calibration 7](#factory-geometric-calibration)

[2.2 Factory Spatial/Resolution Characterization and Calibration
7](#factory-spatialresolution-characterization-and-calibration)

[2.3 Factory Radiometric & Spectral Calibration
7](#factory-radiometric-spectral-calibration)

[2.4 Manufacturer-provided Post-Delivery Calibration Services
7](#manufacturer-provided-post-delivery-calibration-services)

[2.5 Operational Calibration Capabilities
7](#operational-calibration-capabilities)

[3 System Testing and Verification of Performance
7](#system-testing-and-verification-of-performance)

[3.1 Initial System Testing and Proof-of-Design
8](#initial-system-testing-and-proof-of-design)

[3.2 Environmental Testing Plans and Results
8](#environmental-testing-plans-and-results)

[3.3 Standard Article Test Plans and Results Documentation
8](#standard-article-test-plans-and-results-documentation)

[4 Quality Program 8](#quality-program)

[4.1 Design Documentation 8](#design-documentation)

[4.2 Supplier/Materials Qualification
9](#suppliermaterials-qualification)

[4.3 Assembly Documentation: Quality Control History Record
9](#assembly-documentation-quality-control-history-record)

[4.4 Hardware Configuration Management
9](#hardware-configuration-management)

[4.5 Software Configuration Management
9](#software-configuration-management)

[4.6 Warranty and Post-Sale Support 9](#warranty-and-post-sale-support)

[Appendix B: Lexicon 10](#__RefHeading___Toc191276646)

[References 11](#__RefHeading___Toc191276647)

#  {#section .Preface-Heading .unnumbered}

# The USGS Digital Aerial Sensor Certification

## Background and Introduction

This document describes the process by which a manufacturer of a digital
aerial system (the Manufacturer) receives a Type Certification for that
digital aerial system from the United States Geological Survey (USGS).
The Digital Aerial Sensor Type Certification is one part of the overall
USGS Quality Assurance Plan for Digital Aerial Imagery ("the USGS Plan"
or "the Plan"). The USGS Plan is fully described in \[insert reference
here xxxxx\].

The main purpose of Sensor Type Certification is to ensure that the
digital aerial sensor systems, including supporting hardware and
software, are capable of routinely generating mapping-quality aerial
data products when operated in the manner to which they were designed.
This Type Certification intends to assure those who are procuring
digital data that the potential vendor of these data is using a system
that is capable of generating the aerial data products.

## Procedure for Obtaining Type Certification

> Step 1: The process to obtain USGS Type Certification begins when the
> Manufacturer contacts the USGS to request certification. The USGS
> responds and the two parties negotiate the terms and conditions of the
> certification and plan the schedule for the certification effort.
>
> Step 2: After the Certification effort is agreed to by both parties,
> the USGS requests general and technical information from the
> Manufacturer about their digital aerial system. The Manufacturer
> responds to these first inquiries with the information requested.
>
> Step 3: The USGS receives the responses from the Manufacturer, and
> prepares follow-on questions based on the initial answers for
> clarification and more complete understanding of the information
> provided initially. These follow-on questions are delivered to the
> Manufacturer. Depending on the extent and content of these questions,
> this process may be repeated several times until the USGS is satisfied
> that Step 4 should begin.
>
> Step 4: The USGS arranges with the Manufacturer to visit their
> facilities and observe the manufacture and testing of their digital
> aerial system. During the visit the Manufacturer presents and
> demonstrates their system design, quality program, calibration
> methods, and other technical information about their system.
>
> Step 5: Based on the information learned during the visit to the
> Manufacturer's facilities the USGS Team prepares any follow-up
> questions and the Manufacturer provides responses.
>
> Step 6: Upon completion of the above steps and satisfactory
> demonstration of a robust and reliable digital aerial system, the USGS
> issues Type Certification for these systems.

# Information to be Provided by the Manufacturer

In order to fully understand the capabilities and reliability of the
digital aerial system under consideration, the USGS will ask for certain
information from the Manufacturer about their system and about their own
capabilities as manufacturers of such systems.

The USGS also will ask the Manufacturer about the intended operational
environment and maintenance of their system. This knowledge will be used
to better assess the users of these systems in the Data Providers
Certification element of the USGS Plan. A preliminary list of these
questions with explanatory text is given in Appendix A of this document.

The USGS will respect the proprietary rights of the Manufacturer at all
times. The USGS does desire, inasmuch as permitted by the Manufacturer,
to share information with its partners in the Inter-Agency Digital
Imagery Working Group (IADIWG) for their knowledge and benefit. The USGS
asks that the Manufacturer clearly state which information and data
provided to the USGS Team are proprietary and which information can be
shared.

A.  []{#__RefHeading___Toc191276622 .anchor}Required Information

The following is a list of questions and information required by the
USGS. This list is intended to give a reasonably complete example of the
information needed for a generic imaging system. Additional questions
and information requests for specific imaging systems can be expected.

# Technical Questions

> The USGS wants to know the technical design, capabilities, and
> limitations of this system. This includes a strong focus on the normal
> operating environment for the system once it is in use by the
> customer/flier. The USGS needs to understand the basic technical
> design of the system, what capabilities it is intended to have, and
> what operational parameters within which it is intended to operate.
> The USGS is also very interested in how these operational capabilities
> and limitations are conveyed to the customer/flier and how the flier
> is expected to operate and maintain the system in normal use.

## Basic System Design

> Give a brief overview of how the system functions. Include basic
> descriptions of the sensor, the optical path, the detector array(s),
> electronics, and data processing systems. Please include detector
> size/pitch, and effective IFOV, in microradians and any detector
> integration schemes used.
>
> If the system includes IMU & GPS systems they will be described also.
> If the system does not include is own IMU/GPS then describe the
> minimum IMU/GPS inputs and their required quality. If the system does
> not use an IMU/GPS input describe alternatives and their required
> quality.

## Optics

### Basic Optical Layout

> Describe the basic optical layout, including aperture diameter(s),
> effective aperture, focal length(s), filters, detector size(s) and
> orientation(s). Describe also if any of these are adjustable by the
> user.

### Focal Plane:

> Describe the focal plane(s), including composition, size, detector
> pitch, fill factor, and the relative orientation of different color
> bands to each other (i.e. Bayer array, separate color vs. pan arrays,
> etc.)

## Geometry/Geodesy

> This section pertains to the ability of this system to properly locate
> each pixel relative to other pixels in the image (geometry) and
> relative to the actual location on the ground (geodesy).

### Image Distortions

> Describe the intrinsic radial, decentering, and other optical
> distortions that are present in this system.

### Image Distortion Compensation

> Describe how image distortions are compensated for in the production
> of imagery from this system.

### IMU & GPS Accuracies

> Describe the minimum IMU and GPS performance requirements for the
> system to achieve its advertised specifications.

### Multi-imager Systems

> If the system uses multiple lens assemblies, each with their own focal
> planes, describe how the image generated by each focal plane assembly
> is combined with the other images to derive a complete image.

### Forward Motion Compensation

> Describe the capabilities of the system, if any, to compensate for
> forward motion.

### Band-to-Band Registration

> Describe the registration accuracy of individual spectral bands to
> each other. Describe how this band-to-band registration is maintained,
> whether through design or if it is maintained through other methods.

### Geometric/Geodetic Accuracy

> What geodetic accuracy do you claim for your system at the ranges of
> its performance? Please state what you consider to be routinely
> achievable accuracies for minimum and maximum resolution data, along
> with resolution(s) that you expect to be most commonly collected by
> customers. Also, discuss the ground control required to achieve these
> accuracies.

## Radiometry

> This section pertains to the overall sensitivity and response of the
> system to varying light levels and to the spectral performance of the
> system, both in spectral bands and panchromatic bands.

### Definitions

> Please give your definitions of radiometry and describe how they
> relate to your customer's requirements (Mathematical definitions).

### Spectral Response

> How is the "total system" spectral response defined and measured
> (pixel-by-pixel, etc.) lens, filter, etc.?

### Data Quantization

> Explain the data quantization (How many bits are being provided?) and
> any options that are available in the number of bits per pixel.

### Data Compression

> Describe any compression schemes being applied to the data. If there
> is data compression, is it lossless or lossy? If the latter, quantify
> how lossy and what studies have been performed to ascertain their
> effects.

### Dynamic Range

> Describe the dynamic range of the system and how it is defined. (i.e.
> True number of bits, Illumination range, Function of illumination,
> etc. For example, a standard scene could be a 50% Lambertian
> reflective target and a Lambertian shadow with an effective 1%
> reflectance scene with a Midlatitude Summer, 23 km visibility and a
> rural aerosol. What is the typical Signal-to-Noise Ratio (SNR) for
> each band for the 50 % and 1 % reflective target? What are typical DNs
> for these signal levels?)

### Extended Dynamic Range

> Describe the extended dynamic range of the system. What maximum
> illumination/reflectance can be observed without saturating? What are
> the lowest level of illumination where reasonable image quality is
> maintained?

### Flat Fielding

> Explain how flat fielding is typically performed and give the typical
> residuals.

### Linearity

> Describe and quantify any deviation from linearity over the dynamic
> range. Are products produced that represent film equivalents? If so,
> please describe any gamma corrections applied.

### White Balance

> Describe any white balancing that is performed and if it is: 1) How is
> the white balancing performed at different gray levels? 2) What is
> your definition of white balance quality and how is it measured?

### Dark Frames

> Describe any dark frame variation of dark frames as function of
> temperature, ISO setting and integration time, and frame averaging
> etc. How often must dark frames be acquired?

### Absolute Radiometry

> Describe any absolute radiometric calibration done by the manufacturer
> or their agent. Describe calibration procedures and uncertainties. Are
> these results provided to the owner/operator?

## Spatial Performance

### Spatial Resolution General

> Give your definition of spatial resolution and describe how you
> measure that for this sensor. Is there a significant amount of spatial
> and spectral interpolation?

### Maximum Resolution

> What is the highest resolution imagery (smallest ground image feature)
> that can be reliably collected by the purchaser/end-user of the
> system?

### GSD: Ground Sample Distance 

##### What is your definition of GSD?

##### What is it for in-track/ cross-track, each band and on and off axis?

### Relative edge response (normalized edge (0-1) \[slope DN (+.5 pixel) -- DN (-0.5 pixel) including percentage of over and undershoots\] (Should we put this in the lexicon?)

##### What is the slope and over and undershoots for on axis, off axis, each band (9 locations)? Description of measurement of integration time, speed, target size, contrast {#what-is-the-slope-and-over-and-undershoots-for-on-axis-off-axis-each-band-9-locations-description-of-measurement-of-integration-time-speed-target-size-contrast .unnumbered}

### Optical Transfer Function

> What is the system level Optical Transfer Function (OTF), Modulation
> Transfer Function (MTF), Phase Transfer Function (PTF)?

##### On axis, off axis, each band (9 locations)

##### Description of measurements and conditions integration time, speed, target size

### Spatial Measurement Tools and Techniques:

> Describe any special tools or techniques used to characterize actual
> system spatial performance.

### Operational Issues

> Are there any special operational issues that the flyers/operators
> must be aware of when generating imagery at the highest resolution
> possible with this system?

# System Calibration

The USGS is interested in how the digital aerial sensor system is
calibrated, both at the factory and after the system has been delivered
and is operational. The questions below deal with "Factory" calibration,
what is done before delivery; and "Operational" calibration, that which
happens after delivery.

## Factory Geometric Calibration

> Describe the system calibrations that are done to characterize and
> compensate for geometric and geodetic performance. Describe the
> equipment used and the theoretical basis for the tests and
> calibrations. Describe what results of system calibrations are
> delivered to the purchaser/end user.

## Factory Spatial/Resolution Characterization and Calibration

> Describe the system calibrations that are done to characterize and
> compensate for spatial/resolution performance. Describe the equipment
> used and the theoretical basis for the tests and calibrations.
> Describe what results of system calibrations are delivered to the
> purchaser/end user.

## Factory Radiometric & Spectral Calibration

> Describe the system calibrations that are done to characterize and
> compensate for radiometric and spectral performance. Describe the
> equipment used and the theoretical basis for the tests and
> calibrations.
>
> Describe what results of system calibrations are delivered to the
> purchaser/end user.

## Manufacturer-provided Post-Delivery Calibration Services

> Does the manufacturer provide calibration services after delivery? If
> so, describe the calibration services and how they may differ from
> those described above.

## Operational Calibration Capabilities

> Describe any self-calibration capabilities that the owner/operator of
> the system can use after delivery. These can be done either to check
> the stability of the system or to fully calibrate the system.

### Calibration Support Equipment 

> Describe any needed hardware, software, ranges, or capabilities needed
> by the operator to perform operational calibration.

### In-situ Techniques

> Describe any in-situ self calibration techniques that may be used for
> the system, including any benefits, limitations, or special
> considerations that these methods have.

### Frequency of Operational Calibration

> How often does the manufacturer recommend that systems be calibrated
> after delivery?

# System Testing and Verification of Performance

The USGS desires to see how the system has been tested during initial
design and development, and during manufacturing. The USGS also requests
to see independent testing and assessment of the performance and quality
of the products from this digital aerial sensor system.

## Initial System Testing and Proof-of-Design

> Describe, in overview, how the system was initially tested (most
> likely at the prototype or pre-production level) and proven to be
> capable of delivering mapping-quality data. Include efforts taken
> through the design and initial fabrication stages. Include any
> significant testing of subassemblies and their results.

## Environmental Testing Plans and Results

> Describe the environmental tests (temperature, pressure, shock,
> vibration, etc.) that have been conducted to demonstrate the
> capabilities and limitations of the system. Is each system put through
> this testing or were only the initial test articles?

## Standard Article Test Plans and Results Documentation

> Describe how each individual system is routinely tested during the
> manufacturing process, what records are kept of individual system test
> results and performances, and which results, if any, are transmitted
> to the purchaser/end user of the product.

### Independent Assessment of Performance

> For this section, please give reference to testing and analysis of
> system performance performed by external entities, such as
> universities, government organizations, or private industry that is
> not controlled by the Manufacturer. The USGS will review and consider
> these results as part of the Sensor Certification process.

# Quality Program

The Quality Program under which the aerial sensors, supporting systems,
and software were developed is of paramount interest to the USGS. By
"Quality Program" the USGS means the formal plan for maintaining quality
throughout the entire process of designing, building, testing, and
delivering the aerial sensor and its supporting hardware and software.
If your quality program has been certified by an external body, such as
the ISO, please inform the USGS of such certification and also show the
certified plan to the USGS.

Areas of particular interest that the USGS expects to see within the
Quality Program are given below.

## Design Documentation

> Describe the documentation of engineering drawings, specifications,
> and assembly instructions used to build the aerial sensor and
> supporting hardware and software.

## Supplier/Materials Qualification

> Describe how suppliers of critical assemblies and sub-assemblies for
> the system are qualified. This includes hardware and/or software (i.e.
> lenses, detector arrays, processing systems, software, etc.)

## Assembly Documentation: Quality Control History Record

> Describe/demonstrate the documentation used in the assembly and
> testing of the system. Describe the tracking and records collected for
> each system as it moves through the production, testing, and
> calibration process.

## Hardware Configuration Management

> How is the configuration of systems maintained over time? Have there
> been significant configuration changes since this system first went on
> the market? How are changes to the configuration incorporated? Are
> existing systems retrofitted to incorporate new configurations? How
> are customers/owners of systems told of needed changes/upgrades to
> their hardware?

## Software Configuration Management

### Describe how the configuration of software is maintained over time. Describe how changes are proposed, approved, made, tested, and released. Describe how the software has evolved since it first entered the market and how those changes impact users.  {#describe-how-the-configuration-of-software-is-maintained-over-time.-describe-how-changes-are-proposed-approved-made-tested-and-released.-describe-how-the-software-has-evolved-since-it-first-entered-the-market-and-how-those-changes-impact-users. .unnumbered}

## Warranty and Post-Sale Support

> Describe the service and support options available to owners/operators
> of the system, including warranties and any requirements that the
> owner/operator must adhere to maintain the warranty.

B.  []{#__RefHeading___Toc191276646 .anchor}Lexicon

This lexicon defines the intended meanings of terms as they are used in
this document.

**[Customer/End-User]{.underline}** or **[Owner/Operator:]{.underline}**
This refers to the final user/operator of the aerial imaging system
under discussion. Typically this is an aerial services company, although
in some situations the functions of flying and subsequent data
processing are split among separate firms. In such a case both firms
would be considered "end-users" of the respective portions of the system
that they are using.

**[*In-Situ* Calibration:]{.underline}** Refers to techniques that allow
the sensor to derive calibration solutions from data that are collected
under conditions reasonably similar to routine operations. These
techniques may require flying over a surveyed range with recognizable
targets or other special considerations, but are otherwise similar to
ordinary data collection.

**[Multi-spectral Imaging:]{.underline}** Refers to images that are
collected in multiple, distinct, known portions of the spectrum, usually
visible and/or infrared. This is more than simple "color or false color
imaging" which delivers imagery in color but not in a quantified, known,
spectral sampling.

**[Sensor:]{.underline}** This term is used to describe the central
imaging device of the overall sensor system. This is sometimes referred
to as the "camera". Please note that this does not refer exclusively to
the detector array(s). Those are inherent parts of a digital sensor, but
not the complete sensor.

**[Sensor System:]{.underline}** Refers to the entire system sold by the
manufacturer. This can include the sensor (camera), support equipment,
software, and IMU, GPS. Items not sold directly by the manufacturer, but
added on later by the end user (perhaps IMU or GPS) would then not be
considered part of the "sensor system" sold by this manufacturer.

**[Spatial Performance:]{.underline}** This term is used to refer to the
overall system response to an input of varying frequency. While terms
like "Ground Sample Distance (GSD)", "pixel size", or individual Field
of View (IFOV)" are used to describe certain aspects related to this
function, each of those terms is an optical dimensioning and does not
take into account the input stimuli. The best quantifiable measure of
spatial performance for most systems is the Modulation Transfer Function
(MTF).

**[Type Certification:]{.underline}** This certification covers all
identical systems. Thus once a particular make and model digital aerial
system is "type certified", all copies of that item, whether made prior
to certification or following, are also considered to be certified.
However, systems that differ significantly in how they operate or
produce data are not covered under the "type certification" given for a
different model sensor.

[]{#__RefHeading___Toc191276647 .anchor}References

DCN xxxxx The USGS Plan