Package: herbie-data
Herbie: Retrieve NWP Model Data 🏁Herbieis a python package that downloads recent and archived numerical weather prediction (NWP) model output from different cloud archive sources.Its most popular capability is to download HRRR model data.NWP data in GRIB2 format can be read with xarray+cfgrib. Much of this data is made available through theNOAA Open Data Dissemination(NODD) Program (formerly the Big Data Program) which has made weather data more accessible than ever before.Herbie helps you discover, download, and read data from:High Resolution Rapid Refresh (HRRR)|HRRR-AlaskaRapid Refresh (RAP)Global Forecast System (GFS)Global Ensemble Forecast System (GEFS)ECMWF Open Data Forecast ProductsNorth American Mesoscale Model (NAM)National Blend of Models (NBM)Rapid Refresh Forecast System - Prototype (RRFS)Real-Time/Un-Restricted Mesoscale Analysis (RTMA/URMA)Hurricane Analysis And Forecast System (HAFS)📓Herbie DocumentationInstallationThe easiest way to instal Herbie and its dependencies is withCondafrom conda-forge.condainstall-cconda-forgeherbie-dataYou may also create the provided Conda environment,environment.yml.# Download environment filewgethttps://github.com/blaylockbk/Herbie/raw/main/environment.yml# Modify that file if you wish.# Create the environmentcondaenvcreate-fenvironment.yml# Activate the environmentcondaactivateherbieAlternatively, Herbie is published on PyPI and you can install it with pip,butit requires some dependencies that you will have to install yourself:Python 3.8 to 3.11cURLCartopy, which requires GEOS and Proj.cfgrib, which requires eccodes.Optional:wgrib2Optional:Carpenter WorkshopWhen those are installed within your environment,thenyou can install Herbie with pip.# Latest published versionpipinstallherbie-data# ~~ or ~~# Most recent changespipinstallgit+https://github.com/blaylockbk/Herbie.gitCapabilitiesSearch for model output from different data sources.Download full GRIB2 files.Download subset GRIB2 files (by grib field).Read data with xarray.Read index file with Pandas.Plot data with Cartopy (very early development).graph TD;
      d1[(HRRR)] -..-> H
      d2[(RAP)] -.-> H
      d3[(GFS)] -..-> H
      d33[(GEFS)] -.-> H
      d4[(ECMWF)] -..-> H
      d5[(NBM)] -.-> H
      d6[(RRFS)] -..-> H
      d7[(RTMA)] -.-> H
      d8[(URMA)] -..-> H
      H((Herbie))
      H --- .inventory
      H --- .download
      H --- .xarray

      style H fill:#d8c89d,stroke:#0c3576,stroke-width:4px,color:#000000fromherbieimportHerbie# Herbie object for the HRRR model 6-hr surface forecast productH=Herbie('2021-01-01 12:00',model='hrrr',product='sfc',fxx=6)# Look at file contentsH.inventory()# Download the full GRIB2 fileH.download()# Download a subset, like all fields at 500 mbH.download(":500 mb")# Read subset with xarray, like 2-m temperature.H.xarray("TMP:2 m")Data SourcesHerbie downloads model data from the following sources, but can be extended to include others:NOMADSNOAA Open Data Dissemination Program (NODD)partners (i.e., AWS, Google, Azure).[ECMWF Open Data}(https://www.ecmwf.int/en/forecasts/datasets/open-data)Azure storageUniversity of Utah CHPC Pando archiveLocal file systemHistoryDuring my PhD at the University of Utah, I created, at the time, theonly publicly-accessible archive of HRRR data. Over 1,000 research scientists and professionals used that archive.Blaylock B., J. Horel and S. Liston, 2017: Cloud Archiving and Data Mining of High Resolution Rapid Refresh Model Output. Computers and Geosciences. 109, 43-50.https://doi.org/10.1016/j.cageo.2017.08.005In the later half of 2020, the HRRR dataset from 2014 to present was made available through theNODD Program(formerly NOAA's Big Data Program). Herbie organizes and expands my original download scripts into a more coherent package with the extended ability to download data for other models from many different archive sources.I originally released this package under the name “HRRR-B” because it only worked with the HRRR dataset; the “B” was for my first-name initial. Since then, I have added the ability to download RAP, GFS, ECMWF, GEFS, RRFS, and others with potentially more models in the future. Thus, this package was renamedHerbie, named after one of my favorite childhood movie characters.The University of Utah MesoWest group now manages aHRRR archive in Zarr format. Maybe someday, Herbie will be able to take advantage of that archive.How to Cite and AcknowledgeIf Herbie played an important role in your work, pleasetell me about it! Also, consider including a citation or acknowledgement in your article or product.Suggested CitationBlaylock, B. K. (2022). Herbie: Retrieve Numerical Weather Prediction Model Data (Version 2022.9.0) [Computer software].https://doi.org/10.5281/zenodo.4567540Suggested AcknowledgmentA portion of this work used code generously provided by Brian Blaylock's Herbie python package (https://doi.org/10.5281/zenodo.4567540)Thanks for using Herbie, and happy racing!🏁 Brian👨🏻‍💻Contributing Guide/Disclaimer💬GitHub Discussions🚑GitHub Issues🌐Personal Webpage🌐University of Utah HRRR archiveP.S. If you like Herbie, check out my other repos:🌎 GOES-2-go: A python package to download GOES-East/West data and make RGB composites.🌡 SynopticPy: A python package to download mesonet data from the Synoptic API.🔨 Carpenter Workshop: A python package with various tools I made that are useful (like easy funxtions to build Cartopy maps).💬 Bubble Print: A silly little python package that gives your print statement's personality.📜 MET Syntax: An extension for Visual Studio Code that gives syntax highlighting for Model Evaluation Tools (MET) configuration files.Note: Alternative Download ToolsAs an alternative to Herbie, you can usercloneto download files from AWS or GCP. I love rclone. Here is a shortrclone tutorial|Visualize Structure|