GIS In The Rockies, 2020 - Automation Of Spatial Data Validation - NHDPlusHR, NGTOC, USGS
Channel: Greg Cocks
Date: 2020-10-27

The National Hydrography Dataset (NHD) represents the surface water drainage network of the United States with features such as rivers, streams, canals, lakes, ponds, coastline, dams, and stream gauges. The NHD High Resolution, along with the Watershed Boundary Dataset (WBD) and 3D Elevation Program (3DEP) data, is used to create the NHDPlus High Resolution (NHDPlusHR.)
The publicly available spatial data maintain the richness of the NHD complex database model, including multiple feature datasets, feature classes, event feature classes, attribute tables, relationship classes, domains, and feature-level metadata.
After generation, over 220 HU4 NHD large spatial data sets across the contiguous US (and more in Alaska) need to be validated. These same 220+ data sets, as well as Alaska’s, will then be refreshed and then validated on a broad 3-year cycle.
As part of the creation of this inherently spatial dataset, a program of systematic and complete data validation is necessary before it is released to the public. The automation of this validation has been ongoing over the last few years and continues – incrementally replacing the manual processes that had / are being used.
For the expected 60+ Python scripts (integrated into a larger process) that have been and will be written to support this validation, script speed is important in making complete data management and QA/QC across these large datasets feasible – so numpy, pandas and (likely) soon geopandas & (scripted) network tracing are / will be utilised to support this automation.