Package: cmaqsatproc
cmaqsatprocSatellite Processors designed for simple CMAQ comparisons.What can you do?convert L2 or L3 satellite products to L3 on CMAQ grids2-d species like total or tropospheric columnsn-d vairables like averaging kernels and scattering weights.convert CMAQ concentrations to L3-like productsApply satellite averaging kernels to CMAQ concentrations to make satellite-like CMAQApply CMAQ to create alternative air mass factors to make CMAQ-like satellite products.What makes it simple?cmaqsatprochas an easy full suite approachOperates on local files or dynamically finds remote filesUser can specify input files from their disk.Queries NASA's Common Metadata Repository (CMR) or NOAA AWSAllows for spatial subsetting based on a simple box.User can specify the box based on lat/lonThe CMAQ grid can be used to automatically define the box.Provides L2 access as a dataframe or makes Level 3 data as a datasetSimple instructionsare provided to configure Google Colab.Short ExampleOMI NO2 Satellite -- OpenDAP to CMAQ-GridThis example requires OpenDAP access configured on the machine, but does not
require any local files (satellite or CMAQ).import cmaqsatproc as csp

GDNAM = '12US1'
date='2019-07-24'
readername = 'TropOMINO2' # or TropOMIHCHO, VIIRS_AERDB, ...
outpath = f'{readername}_{date}_{GDNAM}.nc'

cg = csp.open_griddesc(GDNAM)
satreader = csp.reader_dict[readername]

l3 = satreader.cmr_to_level3(
    temporal=f'{date}T00:00:00Z/{date}T23:59:59Z',
    bbox=cg.csp.bbox(), grid=cg.csp.geodf, verbose=9
)
l3.to_netcdf(outpath)SNPP VIIRS Deep Blue -- Downloaded files to CMAQ-GridThis example assumes you have downloaded satellite files. The code is largely
the same as the previous. Instead ofcmr_to_level3, it the method usesglobto make a list of files that it passes topaths_to_level3.from glob import glob
import cmaqsatproc as csp

GDNAM = '12US1'
date='2019-07-24'
readername = 'VIIRS_AERDB' # or TropOMIHCHO, VIIRS_AERDB, ...
outpath = f'{readername}_{date}_{GDNAM}.nc'

cg = csp.open_griddesc(GDNAM)
satreader = csp.reader_dict[readername]

paths = sorted(glob('AERDB_L2_VIIRS_SNPP*.nc'))
l3 = satreader.paths_to_level3(
    paths, bbox=cg.csp.bbox(), grid=cg.csp.geodf, verbose=9
)
l3.to_netcdf(outpath)CMAQ NO2 to OMIThis example requires an output from one of the previous exmaples. The data
from the level3 satellite output is combined with CMAQ to make a comparison.import cmaqsatproc as csp
import xarray as xr

GDNAM = '12US1'
date='2019-07-24'
readername = 'TropOMINO2'

satreader = csp.reader_dict[readername]
l3 = xr.open_dataset(f'{readername}_{date}_{GDNAM}.nc')

qf = csp.open_ioapi(f'CCTM_CONC_{date}_{GDNAM}.nc')[['NO2']]
mf = csp.open_ioapi(f'METCRO3D_{date}_{GDNAM}.nc')
qf['DENS'] = mf['DENS']
qf['ZF'] = mf['ZF']
qf['PRES'] = mf['PRES']
# Create satellite according to CMAQ, and CMAQ according to satellite
overf = satreader.cmaq_process(qf, l3)
overf.to_netcdf(f'{readername}_{date}_{GDNAM}_CMAQ.nc')What assumptions are being made?Spatial matching is pretty goodFor satellite products with pixel corners, fractional area weighting is used by default. Other options are avilable.Satellite AveragingKernels are averagedwithin a single day or overpass depending on configuration.within grid cells.CMAQ stratosphere is using one of several methodsRemoved according to the satellite averaging kernel.Prerequisitesnumpyxarraynetcdf4pyprojpandasgeopandash5netcdf is required for s3 supportOpenDAP SupportOpenDAP is supported through standard NetCDF-C support. If authentication is
required, configure.netrcand.dodsrc. The configuration is described
several places. Although urls tend to update, the links below are currently useful:https://disc.gsfc.nasa.gov/data-accesshttps://docs.unidata.ucar.edu/netcdf-c/current/auth.htmlhttps://docs.opendap.org/index.php/DAP_Clients_-_AuthenticationTo summarize those resources, make a user-access-only.netrcfile. Then, make
a.dodsrcfile that points to the.netrcfile and a.urs_cookiesfile.
The commands below achieve this goal, but will overwrite anything you already
have there:touch~/.netrc
touch~/.urs_cookies
chmod0600~/.netrc
cat<< EOF > ~/.dodsrcHTTP.NETRC=${HOME}/.netrcHTTP.COOKIEJAR=${HOME}/.urs_cookiesEOF# where <uid> and <password> are your Earthdata credentialscat<< EOF >> ~/.netrcmachine urs.earthdata.nasa.govlogin <uid>password <password>EOFNotes:I have only been able to make this work if the files are in the user home directory.netcdf4-python version 1.6 has trouble with opendap.Diagramflowchart TB;
    subgraph userinputs [User Options];
    direction TB;
    query_opts(query options);
    localpaths[(local paths\nor urls)];
    grid(CMAQ grid);
    end
    subgraph cmaqsatproc [ ];
    direction TB;
    csplabel[cmaqsatproc];
    cmr_links[[cmr_links]];
    xarray -->to_dataframe[[to_dataframe]]
    xarray[[open_dataset]] -->to_level3[[to_level3]]
    end
    subgraph outputs
    direction LR;
    l3;
    csv;
    end
    subgraph CMR [. Common Metadata Repository .];
    direction TB;
    CMR_API[[CMR API]];
    NASA_DB[(NASA Database)];
    NASA_DB <--> CMR_API;
    end
    grid -->to_level3
    query_opts -->cmr_links;
    cmr_links <-->CMR_API;
    localpaths --> xarray;
    cmr_links -- OpenDAP links--> xarray;
    to_level3 --> l3[\Level3 NetCDF or CSV/];
    to_dataframe --> csv[\Level2 CSV/];
    style query_opts fill:#ff0,stroke:#333,stroke-width:4px
    style userinputs fill:#fff,stroke:#000,stroke-width:1px
    style outputs fill:#fff,stroke:#000,stroke-width:1px
    style grid fill:#ff0,stroke:#333,stroke-width:4px
    style localpaths fill:#ff0,stroke:#333,stroke-width:4px
    style CMR fill:#ccc,stroke:#333,stroke-width:4px
    style cmaqsatproc fill:#cefad0,stroke:#cefad0,stroke-width:4px
    style csplabel fill:#cefad0,stroke:#cefad0,stroke-width:4pxReferencesHenderson, B. H. (2022, October 17). cmaqsatproc v2: Satellite data processing for CMAQ [Poster]. 21st Annual Community Modeling and Analysis System Conference, Chapel Hill, NC.https://www.cmascenter.org/conference/2022/agenda.cfm;direct link