DOI: 10.5061/dryad.fbg79cp80
title: Using optical flow temporal interpolation of satellite imagery to assist multi-sensor global cloud product composites

description: Large domain cloud-product composites, important for climate research and
 operations in civil and military aviation, maritime, and renewable energy
 management, require the synthesis of products retrieved from different
 satellites scanning at different times around the globe.  Common
 cloud-product composites simply use a pixel within the nearest scan time
 (with similar sensor zenith angles) to select as the valid cloud state for
 any given image.  Such an assumption leaves clouds out of valid
 time position, especially when large temporal differences exist between
 the composite valid time and imagery scan time over fast-moving clouds
 (i.e. cirrus).  Within this manuscript, we introduce a method for
 accounting for cloud motions between different scanned imagers before
 composite synthesis, which we term as “Temporal Correction.”  The
 method uses an advanced dense (every image pixel) optical flow retrieval
 technique coupled with a simple, occlusion reasoning warping method
 commonly used for temporal resolution enhancement.  The optical
 flow retrieval was tuned using a large comparison between temporally
 interpolated full-disk geostationary satellite imagery to corresponding
 fine-temporal resolution 1-min scans.  It is demonstrated through
 six separate case studies that the temporal correction methodology
 improves the correspondence between retrieved cloud-top heights from
 various geostationary and low-earth orbiting imagers by ~4.6-13.5% in
 reduction of root mean squared error after such products are parallax
 corrected and remapped to a common rectilinear grid.  Similar
 improvements can be extended to products relevant to 3D cloud
 reconstruction, such as cloud-base height.
description: # Using optical flow temporal interpolation of satellite imagery to assist
 multi-sensor global cloud product composites Dataset DOI:
 [10.5061/dryad.fbg79cp80](10.5061/dryad.fbg79cp80) ## Description of the
 data and file structure Data provided here includes imagery and
 cloud-products derived from the GOES-R Advanced Baseline Imager (ABI), the
 Meteosat-10 Spinning Enhanced Visible and Infrared Imager (SEVIRI), and
 the SNPP Visible Infrared Imaging Radiometer Suite (VIIRS) ascending
 orbits, which our algorithm tracks and corrects for motions to composite
 all on to one seamless grid. ### Files and variables #### File:
 OCTANE_Temporal_Correction_Data_Files.tar.gz **Description:** A tarball
 containing all necessary imagery for GOES ABI to Meteosat SEVIRI and SNPP
 VIIRS comparisons. In the ABI_TO_SEVIRI directory, there are directories
 for the ABI (File Count: 2; imagery netcdfs), ABI_ScanLineTime (File
 Count: 1; netcdf with the scan time of each pixel for ABI), CTH_ABI (File
 Count: 2; ABI cloud products in netcdf form), CTH_HDF (File Count: 2; ABI
 raw CLAVR-x output), SEVIRI (File Count: 3; SEVIRI imagery and cloud
 products in netcdf form), and SEVIRI_HDF (File Count: 3; SEVIRI raw
 CLAVR-x output) for the 20 March 2024 case study in our paper. In the
 ABI_TO_VIIRS subdirectory, there are directories containing ABI (File
 Count: 84; imagery netcdfs), CTH (File Count: 25; raw CLAVR-x output), and
 VIIRS (Filecount: 1; h5 SNPP ascending cloud data products) for the 16
 February 2024 case study in our paper. ### **Individual Files:** #### ABI
 NetCDFs: An ABI file uses the following naming convention: *SE_DSN_PID_Obs
 Start Date & Time_Obs End Date & Time_Creation Date & Time.FE*
 Where * SE = System Environment (OR means Operational) * DSN = Dataset
 Short Name, including Dataset Level (e.g. L1b), Sector (e.g. RadF for Full
 Disk), scan mode (e.g. M6 for mode 6) and channel (e.g. C13 for channel
 13) * PID = Platform Identifier (e.g. G16 for GOES-16) * Obs =
 Observation, with time in YYYYJJJHHMMSSs format (s is 10ths of seconds) *
 FE = File Extension, which is nc for the netcdf files ABI files are read
 using jma_goesread.py with an example to read and plot in
 example_goesread.py NetCDF files can also be visualized using ncBrowse,
 ncview, Panoply, or the Unidata Integrated Data Viewer. #### CLAVR-X HDFs:
 A CLAVR-x file uses the following naming convention:
 clavrx_*SE_DSN_PID_Obs Start Date & Time.FE* Where * SE = System
 Environment (OR means Operational) * DSN = Dataset Short Name, including
 Dataset Level (e.g. L1b), Sector (e.g. RadF for Full Disk), scan mode
 (e.g. M6 for mode 6) and channel (e.g. C13 for channel 13) * PID =
 Platform Identifier (e.g. G16 for GOES-16) * Obs = Observation, with time
 in YYYYJJJHHMMSSs format (s is 10ths of seconds) * FE = File Extension,
 which is hdf for CLAVR-x output CLAVRx HDF (both ABI and SEVIRI) files are
 read using the jma_cthread.py function, with an example to read and plot
 in example_cthread.py . These files can also be converted to netcdf if
 needed using convert_ir.py HDF files can also be visualized using HDFView
 and the Unidata Integrated Data Viewer. #### SEVIRI NetCDFs: The SEVIRI
 NetCDFs used here have the following naming convention: clavrx_*PID*_*Obs
 Start Date & Time*.level2.*FE* Where * PID = Platform Identifier (e.g.
 met10 for Meteosat-10) * Obs = Observation, with time in YYYY_JJJ_HHMM
 format * FE = File Extension, which is hdf for CLAVR-x output SEVIRI
 NetCDFs are read using jct_msgread.py , with an example to read and plot
 in example_sevread.py . NetCDF files can also be visualized using
 ncBrowse, ncview, Panoply, or the Unidata Integrated Data Viewer. ###
 VIIRS H5s: The VIIRS H5 file uses the following naming convention:
 *PID_DSN_Obs Day.FE* Where * PID = Platform Identifier (e.g. SNPP-VIIRS
 for the Suomi National Polar-Orbiting Partnership Visible Infrared Imaging
 Radiometer Suite) * DSN = Dataset Short Name (e.g. composite_asc for
 composite ascending granules from VIIRS) * Obs = Observation, with time in
 YYYYJJJ format * FE = File Extension, which is h5 for composite VIIRS
 output VIIRS H5s are ingested in read_viirs_remap.py , using the
 read_resampled_clavrx_h5 function. H5 files can also be visualized using
 HDFView and the Unidata Integrated Data Viewer. ## Code/software Code
 (along with additional installation instructions) to read, temporally
 correct, and plot the datasets is provided with a separate zenodo tarball
 (OCTANE_Temporal_Correction.tar.gz). Within, the
 abi_seviri_temp_correct.py script temporally corrects and plots the ABI
 and SEVIRI 20 March 2024 case study, and the read_viirs_remap.py script
 temporally corrects the 14 February 2024 case study (this takes longer
 since it must correct several ABI images), which can then be plotted with
 plot_viirs_tc.py. #