AndreaBozzo/ceres-open-data-index huggingface dataset:
Distributed Identification of Top-l Inner Product Elements and its Application in a Peer-to-Peer Network

Inner product computation is an important primitive used in many techniques for feature dependency detection, distance computation, clustering and correlation computation among others. Recently, peer-to-peer networks are getting increasing attention in various applications involving distributed file sharing, sensor networks, and mobile ad hoc networks. Efficient identification of top few inner product entries from the entire inner product matrix of features in a distributed peer-to-peer network is a challenging problem since centralizing the data from all the nodes in a synchronous, communication efficient manner may not be an option. This paper deals with the problem of identifying significant inner products among features in a peer-to-peer environment where different nodes observe a different set of data. It uses an ordinal framework to develop probabilistic algorithms to find top-LaTeX elements in the inner product matrix. These l  inner product entries are important in making crucial decisions about dependency or relatedness between feature pairs, important for a number of data mining applications. In this paper we present experimental results demonstrating accurate and scalable performance of this algorithm for large peer-to-peer networks and also describe a real-world application for this algorithm.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
ADVANCED MICROWAVE SOUNDING UNIT-A (AMSU-A) SWATH FROM NOAA-17 V1

AMSU-A, the Advanced Microwave Sounding Unit, is a 15-channel passive microwave radiometer used to profile atmospheric temperature and moisture from the earth's surface to ~45 km (3 millibars). All orbits beginning in the day (00:00:00 - 23:59:59 UTC) are stored in one daily HDF-EOS file. Each file contains 15 (channel) arrays, as well as corresponding latitude, longitude, and time. AMSU flies on the National Oceanic and Atmospheric Administration (NOAA) polar orbiting spacecraft as part of the National Polar-orbiting Operational Environmental Satellite System (NPOESS). The Third Advanced Microwave Sounding Unit-A was launched on NOAA-17 on 24 June 2002 from Vandenberg AFB, California on a Titan II booster.

Tags: atmosphere, atmospheric-temperature, atmospheric-water-vapor, earth-science, microwave, spectral-engineering
Organization: National Aeronautics and Space Administration
OCO-3 Level 1A collated, parsed, calibration data V11 (OCO3_L1aIn_Pixel) at GES DISC

Version 11 is the current version of the data set. Older versions will no longer be available and are superseded by Version 11.

The Orbiting Carbon Observatory is the first NASA mission designed to collect space-based measurements of atmospheric carbon dioxide with the precision, resolution, and coverage needed to characterize the processes controlling its buildup in the atmosphere. The OCO-3 project uses the LEOStar-2 spacecraft that carries a single instrument. It incorporates three high-resolution spectrometers that make coincident measurements of reflected sunlight in the near-infrared CO2 near 1.61 and 2.06 micrometers and in molecular oxygen (O2) A-Band at 0.76 micrometers. The three spectrometers have different characteristics and are calibrated independently. Their raw data numbers (DN) are delivered correlated in time to the Level 1B process as Level 1A products. Each band has 1016 spectral elements, although some are masked out in the L2 retrieval.This product is the input to the Level 1B process. It is depacketized raw data formatted into a standard granularity with calibrated engineering data (for both science and calibration observations), in the Single-pixel Mode of operation.

Tags: atmosphere, atmospheric-chemistry, earth-science, infrared-wavelengths, spectral-engineering
Organization: National Aeronautics and Space Administration
Design and Fabrication of a Breadboard, Fully Conductively Cooled, 2-Micron, Pulsed Laser for the 3-D Winds Decadal Survey Mission Project

&lt;p&gt;
	Design and fabricate a space-qualifiable, fully conductively cooled, 2-micron pulsed laser breadboard meeting the projected 3-D Winds mission requirements.&lt;br /&gt;
	Utilize improvements in key technologies including high-power, long life space proven 804 nm pump diodes; derated diode operation, and heat pipe conductive cooling.&lt;/p&gt;

Tags: active, nasa-headquarters, project
Organization: National Aeronautics and Space Administration
Global Land Survey

The U.S. Geological Survey (USGS) and the National Aeronautics and Space Administration (NASA) collaborated on the creation of the global land datasets using Landsat data from 1972 through 2008. NASA and the USGS have again partnered to develop the Global Land Survey 2010 (GLS2010), a new global land data set with core acquisition dates of 2008-2011. This dataset consists of both Landsat TM and ETM+ images that meet quality and cloud cover standards established by the earlier GLS collections. Data acquired in 2011 were used to fill areas of low image quality or excessive cloud cover.

Tags: earth-science, land-surface, surface-radiative-properties, topography
Organization: National Aeronautics and Space Administration
NOAA-20 VIIRS Level-2 Regional Triple-window Sea Surface Temperature (SST3) Data, version R2024.0

The Visible and Infrared Imager/Radiometer Suite (VIIRS) is a multi-disciplinary instrument that is being flown on the Joint Polar Satellite System (JPSS) series of spacecraft, including the Suomi National Polar-orbiting Partnership (S-NPP) that launched in October 2011.  JPSS is a multi-platform, multi-agency program that consolidates the polar orbiting spacecraft of NASA and the National Oceanic and Atmospheric Administration (NOAA).  S-NPP is the initial spacecraft in this series, and VIIRS is the successor to MODIS for Earth science data product generation.  VIIRS has 22 spectral bands ranging from 412 nm to 12 nm.  There are 16 moderate-resolution bands (750m at nadir), 5 image-resolution bands (375m), and one day-night band (DNB).

Tags: earth-science, ocean-temperature, oceans
Organization: National Aeronautics and Space Administration
OCO-3 Level 1B calibrated, geolocated science spectra, Forward Processing V10 (OCO3_L1B_Science) at GES DISC

Version 10 is the current version of the data set. Older versions will no longer be available and are superseded by Version 10.

The Orbiting Carbon Observatory -3 (OCO-3) was deployed to the International Space Station in May, 2019. It is technically a single instrument, almost identical to OCO-2.

The Orbiting Carbon Observatory is the first NASA mission designed to collect space-based measurements of atmospheric carbon dioxide with the precision, resolution, and coverage needed to characterize the processes controlling its buildup in the atmosphere.

OCO-3 incorporates three high-resolution spectrometers that make coincident measurements of reflected sunlight in the near-infrared CO2 near 1.61 and 2.06 micrometers and in molecular oxygen (O2) A-Band at 0.76 micrometers. The three spectrometers have different characteristics and are calibrated independently. 

Oxygen-A Band cloud screening algorithm is one of the primary cloud screening tools implemented in the operational OCO processing pipeline. The algorithm was introduced and applied to early GOSAT data  with further analysis performed on OCO-2 simulations.

The OCO ABO2 algorithm employs a fast Bayesian retrieval to estimate surface pressure and surface albedo from high resolution spectra of the molecular oxygen (O2) A-band, near 0.765 µm. The radiative transfer forward model (FM) assumes a clear-sky condition, i.e. Rayleigh scattering only, such that differences between the modeled and measured radiances are apparent when the measurement scene contains cloud or aerosol.

Tags: atmosphere, atmospheric-chemistry, earth-science, infrared-wavelengths, spectral-engineering
Organization: National Aeronautics and Space Administration
JERS-1_LEVEL0

JERS-1 Level 0 Data

Tags: agriculture, biosphere, earth-science, erosion-sedimentation, forest-science, geomorphic-landforms-processes, land-surface, land-use-land-cover, landscape, solid-earth, surface-water, tectonics, terrestrial-ecosystems, terrestrial-hydrosphere, topography
Organization: National Aeronautics and Space Administration
Gamma-Ray Imager Polarimeter for Solar Flares Project

&lt;p&gt;
	We propose here to develop the &lt;em&gt;Gamma-Ray Imager/Polarimeter for Solar flares &lt;/em&gt;(&lt;em&gt;GRIPS&lt;/em&gt;), the next-generation instrument for high-energy solar observations. &lt;em&gt;GRIPS &lt;/em&gt;will provide a nearoptimal combination of high-resolution imaging, spectroscopy, and polarimetry of solar-flare gamma-ray/hard X-ray emissions from ~20 keV to &amp;gt;~10 MeV (see Table 1). The key new technology (already developed for non-solar astrophysics by collaborator Steven Boggs) is that of a 3D position-sensitive germanium detector (3D-GeD) that allows the position and energy deposition of every photon interaction to be recorded individually, even within the same detector. With the 3D-GeDs, &lt;em&gt;GRIPS &lt;/em&gt;can use a new single-grid imaging system, called the Multi- Pitch Rotating Modulator (MPRM), that provides a near-ideal point response function with twice the throughput per cm2 of detector area compared to the rotating modulation collimator (RMC) imaging of &lt;em&gt;RHESSI&lt;/em&gt;. In addition, since Compton scattering (the dominant interaction at &amp;gt;~150 keV) of the incident photons can be tracked by the 3D-GeDs, much of the background can be rejected and the polarization of photons can be determined.&lt;/p&gt;
&lt;p&gt;
	The proposed balloon-borne &lt;em&gt;GRIPS &lt;/em&gt;consists of a Spectrometer/Polarimeter with sixteen 3DGeDs in four stacked 2_2 planes, together with a single rotating MPRM grid placed 8 meters away, to provide a full-Sun field-of-view and angular resolution (12.5-arcsec FWHM at gamma ray energies) sufficient to separate the footpoints of 2.2 MeV sources for almost all flares.&lt;/p&gt;
&lt;p&gt;
	The spacecraft-borne version of &lt;em&gt;GRIPS &lt;/em&gt;is envisioned to have sixty-four 3D-GeDs in four stacked 4_4 planes, together with a single grid ~20 meters away, in a spinning spacecraft. Such a space &lt;em&gt;GRIPS &lt;/em&gt;would provide a tremendous leap forward for high-energy solar physics, with order of magnitude increases in effective area for gamma-ray line (and hard X-ray) imaging, and much better angular resolution and image quality (see Table 1). A space &lt;em&gt;GRIPS &lt;/em&gt;will allow the 0.511 MeV positron-annihilation line and the weaker prompt nuclear de-excitation lines to be imaged for the first time. Together with line shifts and shapes from the simultaneous high-resolution spectroscopy, such an instrument will provide powerful new diagnostics of the flare energy release and particle acceleration process. Furthermore, precise polarization measurements would be obtained for the first time to constrain models and determine beaming of energetic electrons.&lt;/p&gt;

Tags: active, nasa-headquarters, project
Organization: National Aeronautics and Space Administration
National Land Cover Database 2006 (NLCD2006)

National Land Cover Database 2006 (NLCD2006) is a 16-class land cover classification scheme that has been applied consistently across the conterminous United States at a spatial resolution of 30 meters. NLCD2006 is based primarily on the unsupervised classification of Landsat Enhanced Thematic Mapper+ (ETM+) circa 2006 satellite data. NLCD2006 also quantifies land cover change between the years 2001 to 2006. The NLCD2006 land cover change product was generated by comparing spectral characteristics of Landsat imagery between 2001 and 2006, on an individual path/row basis, using protocols to identify and label change based on the trajectory from NLCD2001 products. It represents the first time this type of 30 meter resolution land cover change product has been produced for the conterminous United States.

Tags: earth-science, erosion-sedimentation, geomorphic-landforms-processes, land-surface, land-use-land-cover, landscape, soils, surface-radiative-properties, topography
Organization: National Aeronautics and Space Administration
Sounder SIPS: Suomi NPP CrIMSS Level 3 Comprehensive Quality Control Gridded Monthly CLIMCAPS Full Spectral Resolution V2 at GES DISC

WARNING: To users of the derived product “co_mmr_midtrop” (carbon monoxide mass mixing ratio to dry air [kg/kg] at ~500 hPa). This variable has a significant bias due to a conversion error: the molecular weight of carbon dioxide (CO2, 44.01 g/mol) was used instead of carbon monoxide (CO, 28.01 g/mol). To correct, simply multiply “co_mmr_midtrop” by 28.01/44.01. Alternatively, derive a profile of mass mixing ratio from scratch using the retrieved column density values (“mol_lay/co_mol_lay”) in the Level 2 files. For further questions or concerns please contact the Sounder SIPS at: sounder.sips@jpl.nasa.gov

The CLIMCAPS (Community Long-term Infrared Microwave Coupled Product System) algorithm is used to analyze data from the Cross-track Infrared Sounder/Advanced Technology Microwave Sounder (CrIS/ATMS) instruments, also known as CrIMSS (Cross-track Infrared and Microwave Sounding Suite). The CrIS/ATMS instruments used for this product are on board the Suomi National Polar-orbiting Partnership (SNPP) platform and use the Full Spectral Resolution (FSR) data. The CrIS instrument is a Fourier transform spectrometer with a total of 2211 FSR infrared sounding channels covering the longwave (645-1095 cm-1), midwave (1210-1750 cm-1), and shortwave (2000-2550 cm-1) spectral regions. The ATMS instrument  is a cross-track scanner with 22 channels in spectral bands from 23 GHz through 183 GHz.
 
The CLIMCAPS algorithm uses an Optimal Estimation methodology and uses an a-priori first guess to start the process. A CLIMCAPS sounding is comprised of a set of parameters that characterizes the full atmospheric state and includes a variety of geophysical parameters derived from the CrIMSS data. These include surface temperature and infrared emissivity; full atmosphere profiles of temperature, water vapor and ozone; infrared effective cloud top characteristics; carbon monoxide, methane, carbon dioxide, sulfur dioxide, nitrous oxide, and nitric acid.
 
This monthly one degree latitude by one degree longitude level-3 product starts with level-2 retrieval products with QC values of 0 (best), 1 (good), and 2 (don't use) which are provided for each variable. Comprehensive QC accepts a retrieval if the profile is good to the surface and ensures consistent analysis across all levels and variables.

Tags: air-quality, altitude, atmosphere, atmospheric-chemistry, atmospheric-pressure, atmospheric-radiation, atmospheric-temperature, atmospheric-water-vapor, clouds, earth-science, land-surface, ocean-temperature, oceans, precipitation, surface-radiative-properties, surface-thermal-properties
Organization: National Aeronautics and Space Administration
Anomaly Detection for Complex Systems

In performance maintenance in large, complex systems, sensor information from sub-components tends to be readily available, and can be used to make predictions about the system's health and diagnose possible anomalies. 

 However, existing methods can only use predictions of individual component anomalies to guess at systemic problems, not accurately estimate the magnitude of the problem, nor prescribe good solutions. 

 Since physical complex systems usually have well-defined semantics of operation, we here propose using anomaly detection  techniques drawn from data mining in conjunction with an automated theorem prover working on a domain-specific knowledge base to perform systemic anomalydetection on complex systems.

 For clarity of presentation, the remaining content of this submission is presented compactly in Fig 1.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
ICESat-2 L4 Along-Track Sea Ice Thickness V001

This data set reports daily, along-track winter sea ice thickness across the Arctic Ocean. Sea ice thickness is estimated using ATLAS/ICESat-2 L3A Sea Ice Freeboard (ATL10), Version 5 data and NASA Eulerian Snow On Sea Ice Model (NESOSIM) snow loading.

Tags: cryosphere, earth-science, sea-ice
Organization: National Aeronautics and Space Administration
A DISTRIBUTED PROGNOSTIC HEALTH MANAGEMENT ARCHITECTURE

This paper introduces a generic distributed prognostic health management (PHM) architecture with specific application to the electrical power systems domain. Current state-of-the-art PHM systems are mostly centralized in nature, where all the processing is reliant on a single processor. This can lead to loss of functionality in case of a crash of the central processor or monitor. Furthermore, with increases in the volume of sensor data as well as the complexity of algorithms, traditional centralized systems become unsuitable for successful deployment, and efficient distributed architectures are required. A distributed architecture though, is not effective unless there is an algorithmic framework to take advantage of its unique abilities. The health management paradigm envisaged here incorporates a heterogeneous set of system components monitored by a varied suite of sensors and a particle filtering (PF) framework that has the power and the flexibility to adapt to the different diagnostic and prognostic needs. Both the diagnostic and prognostic tasks are formulated as a particle filtering problem in order to explicitly represent and manage uncertainties; however, typically the complexity of the prognostic routine is higher than the computational power of one computational element (CE). Individual CEs run diagnostic routines until the system variable being monitored crosses beyond a nominal threshold, upon which it coordinates with other networked CEs to run the prognostic routine in a distributed fashion. Implementation results from a network of distributed embedded devices monitoring a prototypical aircraft electrical power system are presented, where the CEs are Sun Microsystems Small Programmable Object Technology (SPOT) devices.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
BUV/Nimbus-4 Level 2 Compressed Ozone Profile Data V005 (BUVN4L2CPOZ) at GES DISC

The Nimbus-4 BUV Level 2 Compressed Ozone Profile Data collection or CPOZ contains total ozone, reflectivities, ozone mixing ratios and layer ozone amounts measured every 32 seconds during the daylit portion of an orbit. Mixing ratios are given at 19 levels: 0.3, 0.4, 0.5, 0.7, 1, 1.5, 2, 3, 4, 5, 7, 10, 15, 20, 30, 40, 50, 70 and 100 mbar. Layer ozone amounts are provided at 12 layers: 0.24, 0.49, 0.99, 1.98, 3.96, 7.92, 15.8, 31.7, 63.3, 127, 253, and 1013 mbar (bottom of layer value). This product is a condensed version of the BUV High-Density Ozone Data Product or (HDBUV).

The data were originally created on IBM 360 machines and archived on magnetic tapes. The data have been restored from the tapes and are now archived on disk in their original IBM binary file format. Each file contains about one day of data (14 orbits). The files consist of data records each with seventy-two 4-byte words. The first record is the header record, followed by a series of data records, and ends with several trailer records that pad out the original blocked records. A typical daily file is about 100 kB in size.

The BUV instrument was operational from April 10, 1970 until May 6, 1977. In July 1972 the Nimbus-4 solar power array partially failed such that BUV operations were curtailed. Thus data collected in the later years was increasingly sparse, particularly in the equatorial region.

This product was previously available from the NSSDC as the Compressed Ozone Profile (CPOZ) Data with the identifier ESAC-00010 (old ID 70-025A-05P).

Tags: atmosphere, atmospheric-chemistry, earth-science
Organization: National Aeronautics and Space Administration
HRIR/Nimbus-2 Level 1 Meteorological Radiation Data V001 (HRIRN2L1) at GES DISC

HRIRN2L1 is the High Resolution Infrared Radiometer (HRIR) Nimbus-2 Level 1 Meteorological Radiance Data (NMRT) product and contains infrared radiances converted to equivalent black-body temperature or "brightness" temperature values. The data, originally written on IBM 360 machines, were recovered from magnetic tapes, also referred to as Nimbus Meteorological Radiation Tapes (NMRT). The data are archived in their original IBM 36-bit word proprietary format, also referred to as a binary TAP file.

The HRIR instrument was designed to perform two major functions: first to map the Earth's cloud cover at night to complement the television coverage during the daytime portion of the orbit, and second to measure the temperature of cloud tops and terrain features. The HRIR instrument was launched on the Nimbus-2 satellite and was operational from May 16, 1966 through November 15, 1966.

Measurements taken during daytime do not reveal true surface temperatures since the radiometer operates in the 3.5 to 4.1 micron region, and reflected solar radiation is added to emitted surface radiation. However, reflected sunlight in this spectral region does not saturate the radiometer output and usable pictures can be made.

This product was previously available from the NSSDC with the identifier ESAD-00108 (old ID 66-040A-03A).

Tags: earth-science, infrared-wavelengths, spectral-engineering
Organization: National Aeronautics and Space Administration