Package: hydrotools.nwm-client
OWPHydroTools :: NWM ClientThis subpackage implements various interfaces to retrieve National Water Model (NWM) data from various sources including Google Cloud Platform, NOMADS, local directories, or a generic web server directory listing. The primary use for this tool is to populatepandas.Dataframeobjects with NWM streamflow data. See theNWM Client Documentationfor a complete list and description of the currently available methods. To report bugs or request new features, submit an issue through theOWPHydroTools Issue Trackeron GitHub.InstallationIn accordance with the python community, we support and advise the usage of virtual
environments in any workflow using python. In the following installation guide, we
use python's built-invenvmodule to create a virtual environment in which the
tool will be installed. Note this is just personal preference, any python virtual
environment manager should work just fine (conda,pipenv, etc. ).# Create and activate python environment, requires python >= 3.8$python3-mvenvenv
$sourceenv/bin/activate
$python3-mpipinstall--upgradepipwheel# Install nwm_client$python3-mpipinstallhydrotools.nwm_client# Install nwm_client with gcp capabilities$python3-mpipinstall"hydrotools.nwm_client[gcp]"UsageThe following example demonstrates how one might usehydrotools.nwm_clientto retrieve NWM streamflow forecasts.CodeRetrieving data from google cloud# Import the nwm Clientfromhydrotools.nwm_clientimportgcpasnwmimportpandasaspd# Instantiate model data servicemodel_data_service=nwm.NWMDataService()# Retrieve forecast data#  By default, only retrieves data at USGS gaging sites in#  CONUS that are used for model assimilationforecast_data=model_data_service.get(configuration="short_range",reference_time="20210101T01Z")# Look at the dataprint(forecast_data.info(memory_usage='deep'))print(forecast_data[['value_time','value']].head())Example output<class 'pandas.core.frame.DataFrame'>RangeIndex: 137628 entries, 0 to 137627Data columns (total 8 columns):#ColumnNon-NullCountDtype---  ------            --------------   -----0   reference_time    137628 non-null  datetime64[ns]1   value_time        137628 non-null  datetime64[ns]2   nwm_feature_id    137628 non-null  int643   value             137628 non-null  float324   usgs_site_code    137628 non-null  category5   configuration     137628 non-null  category6   measurement_unit  137628 non-null  category7   variable_name     137628 non-null  categorydtypes: category(4), datetime64[ns](2), float32(1), int64(1)memory usage: 5.1 MBNonevalue_time  value0 2021-01-01 02:00:00   5.291 2021-01-01 03:00:00   5.252 2021-01-01 04:00:00   5.203 2021-01-01 05:00:00   5.124 2021-01-01 06:00:00   5.03Retrieving data from Nomads# Import the nwm Clientfromhydrotools.nwm_clientimporthttpasnwmimportpandasaspd# Path to server (NOMADS in this case)server="https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/prod/"# Instantiate model data servicemodel_data_service=nwm.NWMDataService(server)# Set reference timeyesterday=pd.Timestamp.utcnow()-pd.Timedelta("1D")reference_time=yesterday.strftime("%Y%m%dT%-HZ")# Retrieve forecast data#  By default, only retrieves data at USGS gaging sites in#  CONUS that are used for model assimilationforecast_data=model_data_service.get(configuration="short_range",reference_time=reference_time)# Look at the dataprint(forecast_data.info(memory_usage='deep'))print(forecast_data[['value_time','value']].head())Example output<class 'pandas.core.frame.DataFrame'>RangeIndex: 137628 entries, 0 to 137627Data columns (total 8 columns):#ColumnNon-NullCountDtype---  ------            --------------   -----0   reference_time    137628 non-null  datetime64[ns]1   value_time        137628 non-null  datetime64[ns]2   nwm_feature_id    137628 non-null  int643   value             137628 non-null  float324   usgs_site_code    137628 non-null  category5   configuration     137628 non-null  category6   measurement_unit  137628 non-null  category7   variable_name     137628 non-null  categorydtypes: category(4), datetime64[ns](2), float32(1), int64(1)memory usage: 5.1 MBNonevalue_time  value0 2021-01-01 02:00:00   5.291 2021-01-01 03:00:00   5.252 2021-01-01 04:00:00   5.203 2021-01-01 05:00:00   5.124 2021-01-01 06:00:00   5.03System RequirementsWe employ several methods to make sure the resultingpandas.DataFrameproduced bynwm_clientare as efficient and manageable as possible. Nonetheless, this package can potentially use a large amount of memory.The National Water Model generates multiple forecasts per day at over 3.7 million locations across the United States. A single forecast could be spread across hundreds of files and require repeated calls to the data source. The intermediate steps of retrieving and processing these files into leanerDataFramemay use several GB of memory. As such, recommended minimum requirements to use this package are a 4-core consumer processor and 8 GB of RAM.DevelopmentThis package uses a setup configuration file (setup.cfg) and assumes use of thesetuptoolsbackend to build the package. To install the package for development use:$python3-mvenvenv
$sourceenv/bin/activate
$python3-mpipinstall-Upip
$python3-mpipinstall-Usetuptools
$python3-mpipinstall-e.[develop]To generate a source distribution:$python3-mpipinstall-Uwheelbuild
$python3-mbuildThe packages generated indist/can be installed directly withpipor uploaded to PyPI usingtwine.