U.S. Department of the Interior

U.S. Geological Survey

Open-File Report 2020–1015

National Geospatial Program

Defining Technology Operational Readiness for the 3D 

Elevation Program—A Plan for Investment, Incubation,

and Adoption

Defining Technology Operational Readiness 

for the 3D Elevation Program—A Plan for 

Investment, Incubation, and Adoption

By Jason M. Stoker

National Geospatial Program

Open-File Report 2020–1015

U.S. Department of the Interior

U.S. Geological Survey
U.S. Department of the Interior

DAVID BERNHARDT, Secretary

U.S. Geological Survey

James F. Reilly II, Director

U.S. Geological Survey, Reston, Virginia: 2020

For more information on the USGS—the Federal source for science about the Earth, its natural and living resources, 

natural hazards, and the environment—visit https://www.usgs.gov or call 1–888–ASK–USGS.

For an overview of USGS information products, including maps, imagery, and publications, visit 

https://store.usgs.gov/.

Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. 

Government.

Although this information product, for the most part, is in the public domain, it also may contain copyrighted materials 

as noted in the text. Permission to reproduce copyrighted items must be secured from the copyright owner.

Suggested citation:

Stoker, J.M., 2020, Defining technology operational readiness for the 3D Elevation Program—A plan for investment, 

incubation, and adoption: U.S. Geological Survey Open-File Report 2020–1015, 7 p., 

https://doi.org/10.3133/ofr20201015.

ISSN 2331-1258 (online)
iii

Contents

Abstract...........................................................................................................................................................1

Introduction.....................................................................................................................................................1

The 3D Elevation Program Operational Readiness Levels ......................................................................3

Operational Readiness Level 1—Prototype Testing Phase ...........................................................3

Operational Readiness Level 2—Provisional/Incubation Phase ..................................................3

Operational Readiness Level 3—Condition-Dependent Operational Phase ..............................4

Operational Readiness Level 4—Operational Phase .....................................................................4

Maturation/Gates...........................................................................................................................................6

To Advance from Operational Readiness Level 1 to Operational Readiness Level 2 ................6

To Advance from Operational Readiness Level 2 to Operational Readiness Level 3 ................6

To Advance from Operational Readiness Level 2/3 to Operational Readiness Level 4 .............6

Implementation Plan—Next Steps .............................................................................................................7

Investment Strategy .............................................................................................................................7

Specification Changes .........................................................................................................................7

Data Dissemination...............................................................................................................................7

References Cited............................................................................................................................................7

Figure

1. The National Aeronautics and Space Administration’s Application Readiness 

Levels schema............................................................................................................................. 2

Table

1. Matrix summary of Operational Readiness Levels ............................................................... 5

Conversion Factors

U.S. customary units to International System of Units

Multiply By To obtain

Area

square mile (mi2) 2.590 square kilometer (km2)

International System of Units to U.S. customary units

Multiply By To obtain

Length

centimeter (cm) 0.3937 inch (in.)

meter (m) 3.281 foot (ft)

Area

square meter (m2) 10.76 square foot (ft2)
iv

Abbreviations

3D three dimensional

3DEP 3D Elevation Program

ASPRS American Society for Photogrammetry and Remote Sensing

ESRB Elevation Specification Review Board

GPS Global Positioning System

GPSC Geospatial Product and Service Contracts

lidar light detection and ranging

ORL Operational Readiness Level

pts points

QL2 quality level 2

RMSE root mean square error

TNM The National Map

USGS U.S. Geological Survey
Defining Technology Operational Readiness for the 3D 

Elevation Program—A Plan for Investment, Incubation, 

and Adoption

By Jason M. Stoker

Abstract

The 3D Elevation Program (3DEP) is an acquisition 

strategy that uses data from commercial remote sensing technologies to create three-dimensional maps of the United States 

and U.S. territories. Currently, light detection and ranging and 

interferometric synthetic aperture radar are the two commercial technologies being used to provide three-dimensional 

information to meet the program’s operational requirements. 

This is because there is not a well-established process for 

vendors of new and novel instruments to know when and how 

3DEP will accept their technologies into the 3DEP portfolio. 

The purpose of this plan is to provide a strategy and rules for 

communication between 3DEP and commercial partners interested in proposing their modalities for use in the program. To 

accomplish this, 3DEP will also consider how it invests in new 

technologies and how it disseminates data to and categorizes 

data for the broader community and the public.

Introduction

The 3D Elevation Program (3DEP) is an acquisition strategy that relies on commercial remote sensing technologies to 

provide three-dimensional (3D) maps of the United States and 

U.S. territories. Currently, light detection and ranging (lidar) 

and interferometric synthetic aperture radar are the two commercial technologies being used to provide 3D information to 

meet the program’s operational requirements.

Since the early 2000s, discrete, multiple-return, linearmode lidar has been the preferred technology for survey-grade 

U.S. Geological Survey (USGS) wide-area mapping, and the 

USGS has developed a specification that is profoundly influenced by the data that these instruments provide (Heidemann, 

2018). Requirements documented in the National Enhanced 

Elevation Assessment were explicitly linked to the prevailing technology at the time. However, the new 3D Nation 

Requirements and Benefits Study is geared toward a more 

technology-agnostic approach. While the National Enhanced 

Elevation Assessment focused on lidar technology, this study 

is more open to new and emerging technologies that could 

meet user needs. As a result, new technologies could meet 

various requirements of the 3DEP across the country that are 

not being met today.

New lidar technologies, such as Geiger-mode lidar and 

single-photon lidar, provide similar information as conventional linear-mode lidar, with some differences in how 

data are processed. Recent analyses of these technologies 

showed the potential and limitations of early data (Stoker and 

others, 2016). 3DEP collected data using these technologies 

in a provisional/incubation mode; however, there was not a 

well-established process for the vendors of these (and other 

untested) instruments to know when 3DEP will accept these 

technologies into its operational portfolio. Similarly, point 

clouds derived from aerial imagery have the potential to collect data analogous to lidar in certain land cover types, such as 

where canopy cover is low (Leberl and others, 2010), but to 

date they have not been considered a viable option for 3DEP 

and have not been tested by the program.

To address this problem, the purpose of this plan is to 

provide a strategy and rules for 3DEP to follow and communicate to commercial partners interested in proposing their 

modalities for use in the program. To accomplish this, 3DEP 

will also need to consider changes in how it invests in new 

technologies and how it disseminates and categorizes these 

data to the broader community and the public.

Adopting new commercial instruments to use for 3DEP 

is an iterative process, where requirements define products 

and information needed from technologies, and technologies 

respond to those needs. Needs are initially defined based on 

the operational technologies that are already available. New 

commercial instruments will need to endeavor to meet all 

these current requirements. During this evaluation process, the 

program revisits current requirements to see if new technologies meet (or exceed) these needs; helps find a path forward 

for receiving standardized, nationally consistent data; and 

makes decisions on whether to adopt or not adopt that modality for future 3DEP use. Because data consistency across the 

country is important, the ability of a proposed commercialized 

technology to be used alongside the currently used technologies will factor into the decision-making process.
2 Defining Technology Operational Readiness for the 3D Elevation Program 

The National Aeronautics and Space Administration has 

defined Application Readiness Levels that describe the various 

stages of an application’s maturity (fig. 1). 3DEP desired to 

develop a similar maturity model to help the program support 

emerging technologies and make decisions on implementing technologies. For 3DEP, the application is the ability to 

collect consistent, systematic, and standardized 3D data that 

meet 3DEP’s operational requirements. At a minimum, the 

technology must (1) be able to simultaneously measure the 

tops of canopies and penetrate vegetation to map bare earth 

elevations and the vegetation subcanopy; (2) meet the 3DEP 

requirements for positional accuracy and point density; (3) be 

interoperable with other sensors; and (4) provide data packaged in a standard, open-source file format. The maturity of 

the program’s technologies will be organized as Operational 

Readiness Levels (ORLs).

The USGS may consider providing a small amount of 

investment in the early phases of technology development, 

similar to the National Aeronautics and Space Administration’s 

Application Readiness Levels 1–5. However, the focus on 

acceptance of new technologies for 3DEP will mainly be in the 

later (operational) phases of functionality, with tests that determine if a commercial technology can operationally meet the 

requirements as defined (and where requirements are defined 

and tested ideally by the current specification). Therefore, the 

burden of capital investment in these early prototype stages 

will be on the entity who is proposing their technology be 

adopted by 3DEP. Although the program may support very 

early prototypes, the focus will be on technologies that are 

ready or nearly ready to be purchased and adopted by the 

private sector firms that collect data for 3DEP.

Figure 1. The National Aeronautics and Space Administration’s Application Readiness Levels (ARL) schema (from 

https://www.nasa.gov/sites/default/files/files/ExpandedARLDefinitions4813.pdf).
The 3D Elevation Program Operational Readiness Levels 3

The 3D Elevation Program Operational 

Readiness Levels

Operational Readiness Level 1—Prototype 

Testing Phase

ORL 1 involves using a new instrument, or an existing, 

established instrument developed for other needs but one not 

typically used in our Geospatial Product and Service Contracts 

(GPSC) that may be appropriate for future consideration in 

commercial 3DEP data collections. Data collected using an 

existing instrument most likely will not satisfy every current 3DEP specification. However, feedback and education 

in this phase will help both the proposer and the 3DEP team 

determine what may need to be adapted in the instrument or 

the specification if potential use is promising (for example, 

the instrument provides a lower cost per square mile, higher 

resolution, better accuracy, more 3D information, or other 

benefits).

Data are usually collected by the principal investigator/

contractor using their own funding, and then provided to 

3DEP to evaluate and provide feedback as to the potential of 

the instrument in meeting program needs. If there is potential 

value to 3DEP, the program could invest support in a research 

and development capacity but 3DEP would not acquire the 

data on its own. At least 10 square miles (mi2) of data should 

be provided for assessment to determine if the instrument can 

move to the next ORL phase. Comparison to survey-grade, 

independent ground truth will be required. The 3DEP team is 

responsible for providing adequate feedback to the proposer 

on instrument deviations and potential adjustments needed 

to meet the current specification. As these instruments are 

being proposed to meet 3DEP’s needs, it is assumed that these 

instrument developers and (or) operators are aware of current requirements and understand and can explain the existing 

differences, such as insufficient accuracy, point density, and 

interoperability with existing operational technology.

Data will only be released at the request of the principal investigator/operator. Data will not be considered 3DEP 

Quality, though it could be included on The National Map 

(TNM; https://www.usgs.gov/core-science-systems/nationalgeospatial-program/national-map) for download under a 

Prototype Testing Phase Data (ORL 1) or similar category. If 

released, metadata will need to describe these data as experimental and not operational 3DEP data and detail any caveats 

on use of the data. If necessary, data could also be protected 

under a nondisclosure agreement or cooperative research and 

development agreement. An assessment with feedback on the 

instrument will ensure that everyone has a common understanding of exactly what the current deviations are from the 

specification.

To progress from ORL 1 to ORL 2, data collected by 

these instruments must at least meet lidar quality level 2 

(QL2) for point density and vertical accuracy (point density 

greater than or equal to [≥] 2 points [pts]/square meter (m2) 

and vertical accuracy less than or equal to [≤] 10 centimeters 

(cm) root mean square error [RMSE] in flat open areas, and 

≤30 cm at the 95th percentile under vegetation) (Heidemann, 

2018). Density and accuracy will be tested using the same 

methodology specified in the most recent version of the USGS 

lidar base specification (Heidemann, 2018). The instrument 

must be able to collect 3D information for top of canopy, 

interior canopy/subcanopy, and bare earth. The specification 

for instruments that collect data under water is still being 

developed, but instruments in general should be able to resolve 

the top water surface, features in the water column, and the 

underwater bare earth surface.

Operational Readiness Level 2—Provisional/

Incubation Phase

ORL 2 involves instruments being used in a provisional/

incubation phase. There is a high likelihood that data collected 

will not meet current 3DEP specifications; however, the 3DEP 

team believes that the technology is mature enough to be considered worth investing in and incubating development. In this 

phase, 3DEP could invest in data collections in targeted areas 

of interest. If funding partners are advocating for the use of the 

technology, 3DEP could support the technology if the partners 

acknowledge the higher level of risk and know that the results 

will not necessarily be directly interoperable with previous and 

adjacent fully operational technologies.

3DEP may invest in acquisitions in this phase as funding allows. Alternatively, the program could accept existing 

data from flights flown for other customers, providing that the 

data collection was done in the spirit of 3DEP’s specification. 

Ideally, acquisitions will be conducted by GPSC contractors; however, being a GPSC contractor will not necessarily 

be a requirement for incubation. During this phase, 3DEP 

will work with the data provider in an iterative process to get 

the data as close to meeting program requirements as possible. Full acceptance of data will not be dependent on full 

adherence to the current specification. A standard assessment template may be developed to make sure everyone has 

a common understanding of exactly what is being tested, 

how it is being tested, and how to advance to the next ORL. 

Comparison to survey-grade, global navigation satellite 

system-derived, ground-truthing data will follow the American 

Society for Photogrammetry and Remote Sensing’s (ASPRS’s) 

Accuracy Standards for Digital Geospatial Data (ASPR, 2015 

and https://www.asprs.org/a/society/committees/standards/

Positional_Accuracy_Standards.pdf) for non-vegetated vertical accuracy and vegetated vertical accuracy. If changes need 

to be made to the specification to advance, the 3DEP team will 

propose changes to the Elevation Specification Review Board 

(ESRB) for debate and approval.

Data for at least 100 mi2 will need to be collected in 

ORL 2 to demonstrate wide-area mapping capability. Systems 

will be evaluated by the condition classes described in the 

following paragraph. A system could skip ORL 3 and go to 
4 Defining Technology Operational Readiness for the 3D Elevation Program 

ORL 4 if at least three separate projects of 100 mi2 for each 

ORL 3 class are collected and the collected data passes all the 

specification compliance checks.

The condition classes, which are derived from classes in 

the latest version of the National Land Cover Dataset, are

1. Open Water (for bathymetry)—Areas of open water, 

generally with less than (<) 25 percent cover of vegetation or soil.

2. Low Intensity Developed—Mixture of constructed materials and vegetation, where impervious surfaces account 

for 0–49 percent total cover.

3. Developed—Areas where impervious surfaces account 

for 50–100 percent of total cover.

4. Barren/Herbaceous/Cultivated/Shrubland—Areas 

dominated by rock, grasses, or crops with low tree cover 

(<20 percent).

5. Forested—Areas dominated by trees greater than 

(>) 5 meters tall and >20 percent total canopy cover.

Open water does not need to be tested in a system that 

is only for topographic data and an instrument can mature 

to ORL 4 for topographic data only; however, an instrument 

designed mainly for bathymetry would need to be tested over 

both land and water to advance to ORL 4.

Data will be released to the public because taxpayer 

dollars will be used to acquire the data, unless otherwise 

stipulated in a written agreement. Data will not necessarily be 

considered 3DEP Quality, however, it could be included on 

TNM for download under a new Provisional/Incubation Data 

(ORL 2) or similar category on the TNM webpage. Inclusion 

of data that meets 3DEP calculations will be on a case-by-case 

basis. Metadata will need to describe these data as not being 

standard operational 3DEP data. Deviations from ORL 4 data 

and caveats on use of the data must be specifically detailed in 

the metadata.

Operational Readiness Level 3—

Condition-Dependent Operational Phase

The ORL 3 specification has been adapted to accommodate technologies and instruments that have proven successful at the ORL 2 level. Provisional/incubation data have been 

accepted and instruments have been given the green light to be 

used operationally; however, they have only been accepted to 

be used in the generalized landcover types that were collected 

and proven in ORL 2. If an instrument did not collect data in a 

condition class, or failed accuracy or density requirements in 

a class, the instrument is not approved for operational use in 

those classes. Restrictions have been put in place for ubiquitous use across any environment in the continental United 

States. Conditions on the ground may be limited until multiple 

collection parameters have proven that the instrument can be 

used anywhere.

Data for at least 100 mi2 for each condition class will 

need to be collected in ORL 3. A project can cover land that 

has all condition classes simultaneously; however, at least 

100 mi2 for each class will be needed to make a valid assessment and demonstrate wide-area mapping capability. The 

condition classes, which are derived from values in the latest 

version of the National Land Cover Dataset, are

1. Open Water (for bathymetry)—Areas of open water, 

generally with < 25 percent cover of vegetation or soil.

2. Low Intensity Developed—Mixture of constructed materials and vegetation, where impervious surfaces account 

for 1–49 percent total cover.

3. Developed—Areas where impervious surfaces account 

for 50–100 percent of total cover.

4. Barren/Herbaceous/Cultivated/Shrubland—Areas 

dominated by rock, grasses, or crops with low tree cover 

(<20 percent).

5. Forested—Areas dominated by trees >5 meters tall and 

>20 percent total canopy cover.

Open water does not need to be included in an instrument 

that is only for topographic data and an instrument can mature 

to ORL 4 for topographic data only. Conversely, an instrument 

designed mainly for bathymetry would need to be tested over 

both land and water to advance to ORL 4.

Comparison to the Global Positioning System (GPS) will 

follow ASPRS’s Accuracy Standards for Digital Geospatial 

Data (2015), with at least 20 static survey-grade GPS check 

points collected for each condition class and at least 10 additional vegetated points in the forested class.

Data will be released to the public because taxpayer dollars will be used to collect data. Source data will be considered 

3DEP Quality and will be included on TNM for download. 

Metadata will describe these systems as ORL 3, detail under 

what conditions they have been approved for use and include 

the caveat that they have not been approved under other 

conditions.

Operational Readiness Level 4—Operational 

Phase

ORL 4 involves instruments that have been proven to 

collect data that meet 3DEP’s requirements in all possible situations over large areas. To become ORL 4, an instrument must 

have proven successful in every defined ORL 3 condition/

landcover type at least three times. Any company proposing 

to use these instruments will be allowed to proceed without 

the need to prove data can meet program requirements. These 

instruments are considered fully operational for 3DEP data 

collection. Any problems in resulting data are assumed to 

be issues in operation, not technology. Current linear-mode, 

discrete-multiple returns instruments being used by GPSC 

contractors that can meet 3DEP’s current specifications are 
The 3D Elevation Program Operational Readiness Levels 5

grandfathered in because the program was designed around the 

capability and availability of these instruments. Quality assurance will still be required and testing of results will need to be 

performed for ORL 4 instruments; however, 3DEP believes 

that thousands of square miles of data can be collected consistently and systematically with these instruments.

Data will be released to the public because taxpayer dollars will be used to collect data. Source data will be considered 

3DEP Quality and will be included on TNM for download. 

Metadata will describe these systems as ORL 4 and explain 

that 3DEP has full confidence in the ability of these instruments to meet program requirements; however, the explanation should include the caveat that there are measured uncertainties inherent in the data. Table 1 provides a summary of 

the ORLs.

Table 1. Matrix summary of Operational Readiness Levels.

[ORL, Operational Readiness Level; 3DEP, 3D Elevation Program; mi2, square mile; GPS, Global Positioning System; ESRB, Elevation Specification Review 

Board] 

ORL 

level Purpose Meets 3DEPspecifications?

Size of

test area

Data disseminated 

through

The National Map

Considered 

3DEP quality?

Generalized gate to advance 

to next ORL

ORL 1 New/novel 

instruments

No At least 10 mi2 Case-by-case No List of improvements/changes 

needed to meet current 

specifications (template to 

be developed)

ORL 2 Provisional/

incubation

Case-by-case At least 100 mi2 Case-by-case Case-by-case Test to GPS checkpoints 

Minimum pass/fail of 

point density and vertical 

accuracy

Approval by ESRB of 

specification changes and 

recording of anomalies 

required

Multiple (at least three) 

acquisitions and successful 

tests

ORL 3 Test various 

conditions

Yes At least 100 mi2

for each condition class

Yes Yes Pass/fail based on GPS checkpoints and adherence to 

specification

Test to GPS for each condition class

Multiple (at least three) 

acquisitions and successful 

tests

ORL 4 Operational Yes Evaluated to 

current 3DEP 

specifications

Yes Yes Not applicable
6 Defining Technology Operational Readiness for the 3D Elevation Program 

Maturation/Gates

Instruments do not necessarily need to progress linearly 

from ORL1 to ORL 4 in order to be accepted for use. A 

mature system with contractor confidence could begin testing 

at ORL 2 if data are collected over large areas using proper 

testing protocols. Current linear-mode, discrete-multiple 

return systems are grandfathered in as ORL 4, since 3DEP 

was designed around the availability and capabilities of these 

systems. After the specification has been refined (if needed), 

systems at lower ORLs have the potential of moving to ORL 4 

after at least three successful demonstrations at ORL 2 (over 

all classes) or ORL 3. Topobathy instruments would currently 

default to ORL 2, with further investigations needed, and a 

topobathy specification to be defined.

To Advance from Operational Readiness Level 1 

to Operational Readiness Level 2

1. At least 10 mi2 of data should be provided.

2. Comparison to survey-grade, independent ground truth 

will be required.

3. Data collected by these instruments must at least produce data that meets QL2 levels of point density and 

vertical accuracy (≥2 pts/m2 and ≤10 cm RMSE in flat 

open areas, and ≤30 cm at 95th percentile under vegetation). Density and accuracy will be tested using the same 

methodology specified in the most recent version of the 

USGS lidar base specification (Heidemann, 2018).

4. For topographic data, the instrument must be able to collect 3D information for top of canopy, interior canopy/

subcanopy, and bare earth.

5. For bathymetric data, the instrument must be able to collect 3D information for water surface, interior features, 

and underwater bare earth. The exact specifications for 

requirements for instruments collecting data under water 

are still being developed.

6. 3DEP will provide feedback to the instrument provider 

or operator through a feedback template as to exactly 

how this system differs from the current specification.

To Advance from Operational Readiness Level 2 

to Operational Readiness Level 3

1. At least 100 mi2 of data will need to be collected for 

evaluation.

2. For topographic data, the system must be able to collect 3D information for top of canopy, interior canopy/

subcanopy, and bare earth.

3. For bathymetric data, the system must be able to collect 

3D information for water surface, interior features, and 

underwater bare earth.

4. Specifications for requirements for systems collecting 

data under water are still being developed.

5. Data must be compared to GPS in accordance with 

ASPRS’s Accuracy Standards for Digital Geospatial 

Data (2015) for non-vegetated vertical accuracy and 

vegetated vertical accuracy. Data collected by these 

instruments must at least produce data that meets QL2 

levels of point density and vertical accuracy. Density 

and accuracy will be tested using the same methodology 

specified in the most recent version of the USGS lidar 

base specification (≥2 pts/m2 and ≤10 cm RMSE in flat 

open areas, and ≤30 cm at 95th percentile under vegetation) (Heidemann, 2018).

6. If changes need to be made to the specification, the 

3DEP team will propose these changes to the ESRB for 

approval.

7. Demonstrated success at least three times in this ORL.

To Advance from Operational Readiness 

Level 2/3 to Operational Readiness Level 4

1. At least 100 mi2 of data will need to be collected for 

each condition class.

2. For topographic data, the instrument must be able to collect 3D information for top of canopy, interior canopy/

subcanopy, and bare earth.

3. For bathymetric data, the instrument must be able to collect 3D information for water surface, interior features, 

and underwater bare earth. For instruments collecting 

data under water, the exact specifications for requirements are still being developed.

4. Data must be compared to GPS in accordance with 

ASPRS’s Accuracy Standards for Digital Geospatial 

Data (2015) for non-vegetated vertical accuracy and vegetated vertical accuracy for each of the four topographic 

condition classes.

5. Data collected by these instruments must at least produce data that meets QL2 levels of point density and 

vertical accuracy. Density and accuracy will be tested 

using the same methodology specified in the most recent 

version of the USGS lidar base specification (≥2 pts/m2

and ≤10 cm RMSE in flat open areas, and ≤30 cm at 

95th percentile under vegetation (Heidemann, 2018).

6. The ESRB must approve of any proposed specification change.

7. Success must be demonstrated at least three times for 

each condition class in ORL 4.
References Cited 7

Implementation Plan—Next Steps

This report details the strategy and components for a 

phased ORL system for 3DEP. The purpose of this plan is to 

provide a strategy and rules for 3DEP to follow and communicate to commercial partners interested in proposing their 

modalities for use in 3DEP. To bring new modalities into the 

program, 3DEP will need to change how it works with new 

technologies and how it disseminates and categorizes its gathered data to the broader community and the public.

To complete this effort, the USGS National Geospatial 

Program and National Geospatial Technical Operations Center 

need to develop an investment strategy, identify allowable 

specification changes, and determine how to disseminate 

its data.

Investment Strategy

In order to incubate and mature new systems, considerations will need to be made for both the acquisition of data 

with new technologies (ORL 1–2) and educating staff who 

will identify the deviations in data progressing from ORL 1 to 

ORL 2 and ORL 3. 3DEP is an operational program that relies 

on ORL 4 predominantly. 3DEP staff will need to help guide 

development of an ORL 1 instrument to an ORL 3, including assessing and proposing changes to the specifications if 

needed. Deviations between proposed technologies and status 

quo will vary widely based on the technology being reviewed.

Specification Changes

As it will be unlikely that a new proposed instrument 

will meet every requirement developed for current ORL 4 

instruments, identifying the deviations and deciding if those 

deviations are acceptable to 3DEP will be very important. The 

ESRB has been established to evaluate and vote on proposed 

changes to the specification. Changes requested to the specification should be completed through the proposed change 

template.

A checklist of every requirement that currently needs 

to be met for a technology to be fully compliant with the 

specification has been developed for the ESRB using SpecX. Spec-X is a guide for exploring information about TNM 

data products (https://usgs-mrs.cr.usgs.gov/SPECX/treeview/

index). Identifying the deviations between the current specification and the proposed technology and proposing changes 

to accommodate these new instruments will be the responsibility of the ORL staff dedicated to help shepherd the new 

technologies through the maturity phases. ESRB will need to 

make final decisions on if the requirements being proposed for 

changes are imperative for quality or help facilitate quality but 

are not imperative.

Changes to the specification or to the ORL evaluation process will undoubtedly evolve as this process is 

tested with real-world instruments. Unexpected issues and 

accommodations will be addressed on a case-by-case basis, 

and changes may need to be made to this report explaining 

how 3DEP will test new instruments in an unbiased, consistent way.

Data Dissemination

A new process for categorization of data will need to be 

implemented and metadata will need to be generated for various ORLs. ORL 1 data will not be used when calculating total 

areal extents of 3DEP data availability; the decision to include 

ORL 2 data in TNM may be made on a case-by-case basis. 

The reasons why each dataset does not meet ORL 4 requirements will need to be documented, such as in a list of every 

attribute missing in the specification checklist or what conditions these systems are (or are not) approved for yet.

If taxpayer dollars are used to collect non-ORL 4 data, 

the data should still be made available to the public but with 

the applicable caveats noted. ORL categories should be added 

to spatial metadata and displaying projects by their respective 

ORL should be possible through TNM. With these safeguards, 

the end user will know which datasets have the full confidence of 3DEP and which are more provisional/experimental 

in nature.

While this document details the initial process, using this 

ORL schema in 3DEP decision making may be adjusted after 

lessons learned using real-world examples.

References Cited

American Society for Photogrammetry and Remote 

Sensing [ASPRS], 2015, ASPRS Positional Accuracy 

Standards for Digital Geospatial Data: Photogrammetric 

Engineering and Remote Sensing, v. 81, no. 3, p. A1–A26, 

accessed February 12, 2020, at https://doi.org/10.14358/

PERS.81.3.A1-A26.

Heidemann, H.K., 2018, Lidar base specification (ver. 1.3, 

February 2018): U.S. Geological Survey Techniques and 

Methods, book 11, chap. B4, 101 p., accessed June 21, 

2019, at https://doi.org/10.3133/tm11B4.

Leberl, F., Irschara, A., Pock, T., Meixner, P., Gruber, 

M., Scholz, S., and Wiechert, A., 2010, Point clouds—

Lidar versus 3D vision: Photogrammetric Engineering 

and Remote Sensing, v. 76, no. 10, p. 1123–1134, 

accessed January 15, 2020, at https://doi.org/10.14358/

PERS.76.10.1123.

Stoker, J.M., Abdullah, Q.A., Nayegandhi, A., and Winehouse, 

J., 2016, Evaluation of single photon and Geiger mode lidar 

for the 3D Elevation Program: Remote Sensing, v. 8, no. 9, 

16 p., accessed January 14, 2020, at https://doi.org/10.3390/

rs8090767.
Stoker—Defining Technology Operational Readiness for the 3D Elevation Program—Open-File Report 2020–1015

ISSN 2331-1258 (online)

https://doi.org/10.3133/ofr20201015