IBM DS3512 Storage Array Local storage for PS701 servers IBM EXP3512 Expansion Array Expansion chassis for IBM DS3512 Storage (2) Dell PowerEdge R610 Servers Oracle Database Servers EMC Storage Array Local Storage for R610 servers Dell M1000E BladeCenter Chassis (8) Dell M610 Blade Servers Ingest, Tailoring, Distribution (2) Dell Fibre Channel Modules SAN/DAS connectivity for M610 servers
MASUDAFILE – This is coefficients file for the Masuda surface emissivity model for ocean. MWNOISEFILE – This is microwave noise file. TC_AMSU – This is the ATMS transmittance coefficient file. CLDAVGFILE – This is the cloud averaging table. IO_NL – This is the input/output namelist file. CLOUDS_NL – This the clouds namelist file. MICROW_NL – This is the microwave namelist file. OZONE_NL – This is the trace gas namelist file. PRO_NL – This is the profile namelist file. TEMP_NL – This is the temperature namelist file. WATER_NL – This is the water vapor namelist file. Below is an example. job_coverage_start=201504292011059 job_coverage_end=201504292011357 SATELLITE_ID=npp NUCAPS_ALL_PRODUCT=NUCAPS_ALL_20150429_2011059_2011357.nc BOUNDING_BOX_PRODUCT=Bounding_Box_20150429_2011059_2011357.txt WGRIB_LOC=/opt/apps/ots/wgrib2/wgrib2
ASSINGED Processing node has picked the production job off of the QUEUE. Transition to COPYINPUT. COPYINPUT Processor has picked up queued job and is copying all data from the SAN to the working directory. Processing node successfully completes copying input data from the SAN to the working directory. Can transition to RUNNING or FAILED. RUNNING All input data has been copied to the working directory and the processor has executed the algorithm driver script. Algorithm returns with a code. Can transition to METADATA or FAILED METADATA Algorithm has returned with a zero code and written output file to the Process Status File (PSF). NDE DHS reads the PSF file names to be ingested. Processing node successfully reads the output file names from the PSF and validates the file name pattern with a registered product. Can transition to FAILED or COPYOUTPUT COPYOUTPUT File names have been read from the PSF file and validated.. Processing node successfully completes copying output data from the working directory to the SAN. Can transition to COMPLETE or FAILED. COMPLETE All output data has been successfully copied from the working directory to the Production Job is done – no transition from this state.
Table 61 DHS Error Messages Message Subsystem Application Reason What to do? executeChecks : Exception thrown Ingest An error occurred during the validation of a product file being ingested. Error could be during metadata extraction or when checksums don’t match. Check to make sure h5dump (or other relevant MD extractors) is installed on the server. Make sure (manually) that the file is not corrupt. If it is determined to be corrupt, report it to the Data Provider. CRC checksums did not match Ingest Checksum received in PAF did not match with the one generated by the App for the file being ingested Notify Data Provider of the problem. : Unable to retrieve PAF data from message Ingest PAF file (*.crc, etc.) was corrupt or Same as above. executeChecks : Filename is NULL Ingest The PAF file (.crc, etc) is incomplete and have a File name in it Same as above. Error while initializing variables Ingest LoopName attribute may not have been defined in the Check to make sure LoopName attribute in properly defined in the ESB configuration file
Action definition (in jboss-esb.xml) (jboss-esb.xml) of the Ingest app. Exception caught updating heartbeat Ingest Ingest app is unable to update the DB with the Heartbeat reading due to some SQL/DB error. Check to make sure DB is functioning normally. persistMetadata : Exception thrown, moving file from landing zone to failed dir Ingest Ingest app failed to extract metadata and/or persist it in database Same as in row 1 above. homeFile : Exception thrown routing message to Ingest JMS bridge is not working. This bridge connects Ingest and Factory apps and Ingest uses it to send messages over it to Factory app. Check to make sure there are no errors during Ingest App deployment in JBoss server log. Check to make sure at least one Factory server is running. : Unable initialize IR/Resources objects Ingest Ingest app unable to retrieve rows from IngestRequestLog table for a given IR_ID. May be a DB error or the IRB and IRL Make sure the DB is functioning normally. If it is, a TAL must determine the cause of the inconsistencies between IRL and IRB table.
incompatible with known format(s). homeFile : Exception caught (moving file); Ingest Ingest app unable to move the file to Products ‘home’ directory after successful MD extraction and persistence. Make sure SAN (Product Home folder) is OK. : Product Unknown Ingest The product file being ingested is of unknown product. There DHS database doesn’t have a Product registered with the same pattern as the product file being ingested Make sure DHS database is properly registered with all the products. If the products registration fine, then we may have received a file that is not associated with any registered products. : Multiple product dfs found Ingest Ingest app found there are multiple products registered in DB that has same filename pattern as the file being ingested. Correct the Products registration in DB. Make sure there the filename pattern in unique in ProductsDescriotion table of DHS DB. Bridge Failed to set up connections Ingest JMS bridge between Ingest Make sure the specified Factory
and Factory is not correctly deployed. servers in the bridge configuration are up and running. createJobs: Exception detected Dist Factory Error while initializing variables Dist Factory, PGS Factory, PGS Processor, DIST Processor LoopName attribute may not have been defined in the Action definition (in jboss-esb.xml) to make sure LoopName attribute in properly defined in the ESB configuration file (jboss-esb.xml) of the app. while checking active flag Dist Factory, PGS Factory, PGS Processor, DIST Processor Apps unable to find the relevant loop ‘Active’ in table. Make sure there are ‘Active’ flags defined in the ConfigurationRegistry table for all the loops in various apps. : Exception occurred PGS Factory App unable to execute stored proc to retrieve the completed Make sure the Stored Proc is properly registered in the DB and DB is functioning normally.
May be a DB error. getJismoContents - Exception occurred PGS Factory App unable to execute stored proc to retrieve Job Spec Inputs. Same as above. - Exception occurred PGS Factory App unable to execute stored proc to retrieve the File accumulation thresholds for a Same as above. : Exception detected PGS Factory App unable to create (insert into DB) a Job Spec Input for a Make sure DB is OK. If it is, then follow the Exceptin stack trace to see which specific SQL error caused this failure. updateJobStatus : Exception while updating job status DIST,PGS Factory App unable to update the Job status of a ProductionJob Same as above.
: Exception caught DIST ,PGS Factory Apps unable to compute a checksum for a specified file. Or the app may be unable to create a checksum file. Make sure the file in question (for checksum is being computed) is valid and exists. Also, make sure relevant jar files (used for checksums) exists in Jboss environment. Also, make sure the folder in which the checksum file is created exists. : exception encountered DIST Factory Same as for MD5 checksum. Same as above. : exception encountered DIST Factory Same as above for CRC checksum. Same as above. : Exception caught DIST Factory App is unable to compress the file(s). Make sure file(s) being compressed exists and are valid. If they are, follow the exception’s stack trace in the log file and determine the exact cause of the Exception. : Exception caught creating NR DIST Factory App unable to create (insert into DB) a Make sure the DB is functioning normally. If it is, follow the
for a given Distribution Job after it is executed. Exception’s stack trace to determine the root cause of the Exception. NO Notification Request created for job id. The associated subscription has NO notification type. DIST Factory The relevant subscription (for which a job has been completed) has no Notif type defined. Make sure a Notif type is defined for subscription in question. See appropriate documentation (User Manual) for different types of valid Notifs. DistJobStatus : Exception caught DIST Factory The App is unable to update the status of a specified Distribution job. Usually, these updates happen as Job is moved from one state to another. Make sure the DB is functioning normally. If it is, follow the Exception’s trace to determine the root cause of the problem. : Exception occurred DIST Factory Same as above but for a NotificationJob. Same as above. : Exception occurred PGS Processor The app is unable to copy the files (may be due to Make sure the files in question exist in Product ‘home’
problems with source and/or target folders) to PGS Job staging area. This area is first populated with the relevant input files required for executing the algorithm. directory (from which files are being copied from). Make sure the target directory can be created OK. If everything looks good, then follow the Stack trace of the exception logged to determine the root cause of the problem. Exception scnning PSF file for output files PGS Processor The app is unable to scan the PSF file and/or create the list of output files from PSF file. PSF (Process Status File) file is generated by the Science algorithm. It lists all the Product files that have been generated by Science algorithm and are required to be ingested. Make sure the PSF file is properly generated by Science Algorithm. If it is, then follow the Exception Stack trace to determine the root cause of error/exception. exception occurred PGS Processor This is an internal app error message logged when it is unable Follow the Exceptions stack trace to determine the root cause of the error. In
Exception retrieving jobs list PGS Processor This occurs when the app is unable to create a list of various Job class/size type and their number from service. This info is used to retrieve a specific number of jobs of each class/size type. Make sure the Job Box data is properly defined for the processor node. If it is, then an internal error might have caused this and may require a server restart to resolve it. Exception occured in service RunJobSvc: UpdateJobStatus PGS Processor The app is unable to update the status of a Production job. The status is updated many times as it changes from one state to another. Make sure the DB is functioning normally. If it is then follow the exception’s stack trace to determine the root cause. Exception retrieving pid for algorithm command PGS Processor The app is unable to retrieve the PID (Process ID) of the algorithm process launched. Determine the root cause by following the Exception’s stack trace. Exception invoking stored proc PGS Processor The app is unable to execute the Make sure stored proc is registered in DB and
stored proc to retrieve Productionjobs of a specific type. DB is functioning normally. notifySubscriber : FTPS Push Return not OK DIST Processor runLoop:: Exception retrieving jobs list DIST Processor The app is unable to execute stored proc to retrieve distribution jobs from DB. Make sure stored proc is registered in DB and DB is functioning normally. Exception invoking FTPS Service DIST Processor This error occurs when messages are not able to be delivered to service that actually does the FTPs pushes. Make sure the Dist processor app is deployed without any errors. If needed, redeploy the app. runLoop : Exception invoking Notification Service DIST Processor This occurs when messages are not able to be delivered to the service that notifies Same as above.
subscriber’s external host during the ftps transfers of the files. ftps connections with the specified authentication. If everything is registered correctly in DHS database, then contact the concerned person on the subscriber’s side (and NDE side) and report the issue. : not connect to server DIST Processor Same as above Same as above transferFiles : Login was unsuccessful DIST Processor The app is unable to successfully into subscriber’s ftps server. Same as above transferFiles : Ftps Transfer failed DIST Processor FTPS transfer of onto subscriber’s ftps server has failed. Same as above. transferFiles : Ftps Rename failed DIST Processor The rename (post-transfer) of files failed on subscriber’s ftps server Same as above.
transferFiles : DIST Processor The ftps connection to subscriber’s server (where files are being pushed to) has been closed unexpectedly. Same as above. transferFiles : DIST Processor The error is logged when the data files are unable to be steamed (transferred) onto the subscribers ftps server. Usually occurs when data started to flow over ftps and then stopped for some reason before entire file is uploaded. Same as above. transferFiles : IOException DIST Processor This error is logged when IO problems occur during ftps transfer on the subscriber’s server where files are being pushed to. Same as above
path in the PCF file, have never been created, or was deleted prematurely. NUCAPS development lead. Error in $SCRIPT: the directory defined by $DIR does not exist. The directory may not exist or have a wrong path in the PCF file. Or there was an unknown error in the OS. If this occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: failed to open $FILE The script failed to open the file or create a new file. It may have a wrong path in the PCF file, have never been created, or was deleted prematurely. Or there was an unknown error in the OS. If this occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: failed to copy the $FILE to $TARGET. Failed to copy the file to the target. The file may not exist or the target file name may be wrong. Or there was an unknown error in the OS. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: failed to move $FILE to $TARGET. Failed to rename the file. The file may not or the target file name may be wrong. Or there was an unknown error in the OS. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: At least 1 of the 4 required ATMS files is not present. The script will skip ATMS data files and process CrIS data files only. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: $ITEM are less than or equal to zero. The script read a wrong $ITEM value from the PCF file. It needs to be a positive integer. If this error occurs, this error message may be forwarded to the
NUCAPS development lead. Error in $SCRIPT: APOD is undefined. The script read a wrong APOD value from the PCF file. It needs to be either “yes” or “no”. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: a $ITEM string is undefined in PCF. The $ITEM may not or have a wrong path in the PCF file. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: $PROGRAM returned an error. [See $FILE] There was an error running the program or script. A log file will be appended providing additional details. If this error occurs, this error message may be forwarded to the NUCAPS development lead. in $SCRIPT: Could NOT make link to $FILE. Failed to make a soft-link to the file. The file may not exist or have a wrong path. Or there was an unknown error in the OS. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: Could NOT run $PROGRAM. Failed to run a wgrib command. The command may not exist, have a wrong path, or have wrong input parameters. If this error occurs, this error message may be forwarded to the NUCAPS development lead. Error in $SCRIPT: $ITEM retrieval product will not be generated Failed to generate the EDR or CCR retrieval product. If this error occurs, this error message may be forwarded to the NUCAPS development lead.
This is the PCS statistics monitoring file. This is to be distributed for SDR monitoring at OSPO. Text file 0.0015 MB/file 2700 files/day NUCAPS-EDR-MONITORING_ This is the retrieval monitoring output file. Text file 0.078 MB/file 2700 files/day NUCAPS EDR for for 9 sectors netCDF4 0.215 MB/file 1648 files/day The output CrIS full spatial resolution BUFR file converted from NUCAPS netCDF4. BUFR 0.8 MB/file 2700 files/day The output CrIS full spatial resolution BUFR file converted from NUCAPS netCDF4. BUFR 2.8 MB/file 2700 files/day System Level Design and Data Flow The NUCAPS system is developed to run on the Red Hat Enterprise Linux (16 CPUs – 2GB/CPU) hardware platform. This is the hardware environment on which NDE products will be generated for NPP as specified in the NDE Critical Design Review (CDR) document (NDE, 2007). The system units and documentation will be delivered in a Delivered Algorithm Package (DAP) to the NDE Science Algorithm Development and Integration Environment (SADIE). On
airs_17boxcar_03.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_02.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_14.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_13.txt An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_12.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_11.txt An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_10.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_09.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static
airs_17boxcar_08.txt An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_07.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_17.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_16.txt An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static airs_17boxcar_15.txt An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static cris_17boxcar.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static cris_17boxcar_12.txt An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static cris_17boxcar_11.txt Input An ancillary data file provided by the NUCAPS delivery needed by the OLR code. Static
forecast GRIB file. The first static inputs are the Retrieval and CCR CDL template files. These input static files define the structure of the output netCDF4 Retrieval and CCR product files. The Retrieval unit runs a retrieval algorithm executable. This executable requires a number of static retrieval resource files. There are three output files, the netCDF4 Retrieval and CCR output files, and the run log for the entire unit. Table 74 Retrieval Unit Data Flows File Input/ Output Source Description State Input NDE The Process Control File supplied by the NDE PGM. Input unit) The spatially and temporally collocated CrIS, ATMS, and VIIRS granule data files. CrIS radiances have been apodized. gfs.t??z.pgrbf?? Input NCEP The GFS 6-hour forecast file in GRIB2 format Input unit) This is an internal metadata file containing the bounding box and ascending/descending status for the current granule. This file is required for the metadata. nucaps_l2.nc Input STAR This is the netCDF4 retrieval output template file. Static nucaps_ccr.nc Input STAR This is the netCDF4 CCR output template file. Static
Input STAR This is the netCDF4 CCR archive output template file. Static NUCAPS retrieval metadata template Input STAR This is the granule metadata template for the NUCAPS retrieval product in netCDF4. Static NUCAPS CCR metadata template STAR This is the granule metadata template for the NUCAPS cloud-cleared radiance product in netCDF4 Static Input STAR Tuning coefficient file. Static Input STAR Tuning mask file. Static cris_rtaerr_v10a.asc Input STAR Radiative transmittance file. Static airs_olr.dat Input STAR Radiative transmittance coefficients to compute Outgoing Longwave Radiation Static tr_atms_new.dat Input STAR Most recent version of ATMS transmittance coefficients Static binary.trcoef.cris.v10a Input STAR Post-flight CrlS RTA coeffieicnets Static cris_v10a.t1 Input STAR CrIS cloud averaging table Static jpl_100.inp Input STAR Ensemble error estimate of climatology Static