AndreaBozzo/ceres-open-data-index huggingface dataset:
OCO-3 Level 1A collated, parsed, science or calibration data, Retrospective Processing V11r (OCO2_L1aIn_Sample) at GES DISC

Version 11r is the current version of the data set. Older versions will no longer be available and are superseded by Version 11r.

The Orbiting Carbon Observatory is the first NASA mission designed to collect space-based measurements of atmospheric carbon dioxide with the precision, resolution, and coverage needed to characterize the processes controlling its buildup in the atmosphere. The OCO-3 project uses the LEOStar-2 spacecraft that carries a single instrument. It incorporates three high-resolution spectrometers that make coincident measurements of reflected sunlight in the near-infrared CO2 near 1.61 and 2.06 micrometers and in molecular oxygen (O2) A-Band at 0.76 micrometers. The three spectrometers have different characteristics and are calibrated independently. Their raw data numbers (DN) are delivered correlated in time to the Level 1B process as Level 1A products. Each band has 1016 spectral elements, although some are masked out in the L2 retrieval.This product is the input to the Level 1B process. It is depacketized raw data formatted into a standard granularity with calibrated engineering data (for both science and calibration observations), in the Sample Mode of operation.This is the retrospective processing where the calibration data is estimated from the full timeseries of data (before, during, and after the measurements), and is expected to be of slightly higher quality.

Tags: atmosphere, atmospheric-chemistry, earth-science, infrared-wavelengths, spectral-engineering
Organization: National Aeronautics and Space Administration
sequenceMiner algorithm

Detecting and describing anomalies in large repositories of discrete symbol sequences.

 

**sequenceMiner has been open-sourced! Download the  file below to try it out.**

 

sequenceMiner was developed to address the problem of detecting and describing anomalies in large sets of high-dimensional symbol sequences. sequenceMiner works by performing unsupervised clustering (grouping) of sequences using the normalized longest common subsequence (LCS) as a similarity measure, followed by a detailed analysis of outliers to detect anomalies. sequenceMiner utilizes a new hybrid algorithm for computing the LCS that has been shown to outperform existing algorithms by a factor of five.

sequenceMiner also includes new algorithms for outlier analysis that provide comprehensible indicators as to why a particular sequence was deemed to be an outlier. This provides analysts with a coherent description of the anomalies identified in the sequence, and why they differ from more “normal” sequences.

 

sequenceMiner was developed with funding from the NASA Aviation Safety Program.  In the commercial aviation domain, sequenceMiner can be used to discover atypical behavior in airline performance data that may have possible operational significance for safety analysts.  But because the sequenceMiner approach is general and not restricted in any way to a domain, and these algorithms can be applied in other fields where anomaly detection and event mining would be useful.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
NASA Landsat Data Collection

The NASA Landsat Data Collection (NLDC) is a compilation of Landsat multispectral scanner (MSS) scenes and Landsat thematic mapper (TM)  scenes. This compilation of scenes represents data collections from four  distinct projects including:  (1) the Global Change Landsat Data Collection  (GCLDC);(2) the Humid Tropical Forest Project (HTFP) collection of source  scenes and products; (3) a collection of data from the Committee on  Environment and Natural Resources Research [formerly the Committee on Earth and Environmental Sciences (CEES)] that is historically referred to as  the CEES collection; and (4) ongoing Landsat data purchases by NASA-funded investigators, starting with the 1996 fiscal year.  The NLDC scenes have been screened for cloud cover and band quality resulting in a high  grade,high quality data compilation. The GCLDC collection contains Landsat TM scenes that were purchased  by NASA from Space Imaging, formerly the Earth Observation Satellite  Company,under a special agreement to promote the use of shared data in  global change research.  The HTFP, the largest component of NASA's Landsat Pathfinder Program, contains Landsat MSS and TM scenes collected  over the past 20 years.  The goal of the HTFP is to globally map  deforestation in the humid tropical forests.  The CEES collection is the result of an effort to coordinate data needs among several Federal agencies (e.g.,Environmental Protection Agency, Department of the Interior agencies, National Oceanic and Atmospheric Administration, Department of Defense).  These Landsat TM scenes were collected for a variety of research projects.  Ongoing NASA purchases of Landsat TM data support NASA scientists and their affiliated researchers in programs and projects including the NASA Research and Analysis Program; the Global Land Cover Test Sites Project; the HTFP, the  International Biosphere-Geosphere Programme, the NASA Applications Program; and the Landsat-7 Science Team.

Tags: earth-science, infrared-wavelengths, land-surface, land-use-land-cover, spectral-engineering, visible-wavelengths
Organization: National Aeronautics and Space Administration
Daily Historical Water Balance Products for the CONUS

This dataset provides daily historical Water Balance Model outputs from a Thornthwaite-type, single bucket model.  Climate inputs to the model are from GridMet daily temperature and precipitation for the Continental United States (CONUS). The Water Balance Model output variables include the following: Potential Evapotranspiration (PET, mm), Actual Evapotranspiration (AET, mm), Moisture Deficit (Deficit, mm), Soil Water (soilwater, mm), Runoff (mm), Rain (mm), and Accumulated Snow Water Equivalent (accumswe, mm). The dataset covers the period from January 1 to December 31 for years 1980 through 2023 for the CONUS. Water Balance Model variables are provided as individual files, by variable and year, at a 1 km x 1 km spatial resolution and a daily temporal resolution. Data are in a North America Lambert Conformal Conic projection and are distributed in a standardized Climate and Forecast (CF)-compliant NetCDF file format.

Tags: atmosphere, atmospheric-water-vapor, earth-science, land-surface, precipitation, soils, surface-water, terrestrial-hydrosphere
Organization: National Aeronautics and Space Administration
Model-Based Fault Detection and Diagnosis System for NASA Mars Subsurface Drill Prototype

The Drilling Automation for Mars Environment (DAME) project, led by NASA Ames Research Center, is aimed at developing a lightweight, low-power drill prototype that can be mounted on a Mars lander and be capable of drilling down several meters below the Mars surface for conducting geology and astrobiology research. The DAME drill system incorporates a large degree of autonomy - from quick diagnosis of system state and fault conditions to making the appropriate recovery actions - while also striving to achieve as many of the operational objectives as possible. This paper outlines, on a general level, the overall DAME architecture, equipment, and autonomy package. The main focus, however, is on describing the model-based fault detection and diagnosis system, including the modeling approach, the fault modes handled, and the diagnostic algorithms. The results of the latest field tests, conducted in 2006 in Haughton Crater on Devon Island (a Mars analogue site in Canadian Arctic), are also discussed.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
Sentinel-3B OLCI Level-3M Global Mapped Earth-observation Reduced Resolution (ERR) Diffuse Attenuation Coefficient for Downwelling Irradiance (KD) - Near Real-time (NRT) Data, version R2022.0

The Ocean Biology DAAC produces near real-time (quicklook) products using the best-available combination of ancillary data from meteorological and ozone data. As such, the inputs and the calibration used are less than optimal. Quicklook products provide a snapshot of the data during a short time period within a single orbit.

Tags: atmosphere, atmospheric-radiation, earth-science, ocean-optics, oceans
Organization: National Aeronautics and Space Administration
Physics Based Electrolytic Capacitor Degradation Models for Prognostic Studies under Thermal Overstress

Electrolytic capacitors are used in several applications rang- ing from power supplies on safety critical avionics equipment to power drivers for electro-mechanical actuators. This makes them good candidates for prognostics and health management research. Prognostics provides a way to assess remaining use- ful life of components or systems based on their current state of health and their anticipated future use and operational con- ditions. Past experiences show that capacitors tend to degrade and fail faster under high electrical and thermal stress condi- tions that they are often subjected to during operations. In this work, we study the effects of accelerated aging due to thermal stress on different sets of capacitors under different conditions. Our focus is on deriving first principles degra- dation models for thermal stress conditions. Data collected from simultaneous experiments are used to validate the de- sired models. Our overall goal is to derive accurate models of capacitor degradation, and use them to predict performance changes in DC-DC converters.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
LRIR/Nimbus-6 Level 2 Inverted Profiles of Temperature and Ozone V001 (LRIRN6L2IPAT) at GES DISC

LRIRN6L2IPAT is the Nimbus-6 Limb Radiance Inversion Radiometer (LRIR) Level 2 Inverted Profiles of Temperature and Ozone data product. The product contains daily profiles of temperature and ozone concentration profiles that were inverted from radiances measured in four spectral regions: two in the 15 micron carbon dioxide band; one in the 9.7 micron ozone band; and one located in the rotational water vapor band (23 to 27 microns). The calibrated radiances are also included in this product. There are a maximum of 13 orbits per day each with up to 115 profiles per orbit.

LRIR is a limb profiler with spatial coverage from latitude -64 to +84 degrees. Vertical profiles are provided at 17 standard pressure levels (from 100 to 0.1 mbar, i.e., from 15 to 64 km) with about 1.5 km vertical resolution. The instrument operated successfully and data are available from 20 June 1975 to 6 January 1976. After this, the detector temperature began to rise rapidly, and the instrument was turned off. The principal investigator for the LRIR experiment was Dr. John Gille from NCAR.

This product was previously available from the NSSDC with the identifier ESAD-00037 (old ID 75-052A-04A).

Tags: atmosphere, atmospheric-chemistry, atmospheric-temperature, earth-science, infrared-wavelengths, spectral-engineering
Organization: National Aeronautics and Space Administration
Terra MODIS Level-2 Regional 4µm Day/Night Sea Surface Temperature (SST4) - Near Real-time (NRT) Data, version R2019.0

The Ocean Biology DAAC produces near real-time (quicklook) products using the best-available combination of ancillary data from meteorological and ozone data. As such, the inputs and the calibration used are less than optimal. Quicklook products provide a snapshot of the data during a short time period within a single orbit.

Tags: earth-science, national-geospatial-data-asset, ngda, ocean-temperature, oceans
Organization: National Aeronautics and Space Administration
nu-Anomica algorithm

One-class nu-Support Vector machine (SVMs) learning technique maps the 
input data into a much higher dimensional space and then uses a small 
portion of the training data (support vectors) to parametrize the 
decision surface that can linearly separate nu fraction of training 
points (labeled as anomalies) from the rest. The exact solution of 
standard one-class nu SVMs assigns (at least) nu fraction of training 
points as support vectors. However some of these support vectors may be 
unnecessary or redundant. Hence the computational issue turns alarming 
especially when SVMs based novelty detectors with nonlinear kernels are 
trained on data sets of huge size. The proposed nu-Anomica algorithm can 
solve this problem. The idea is to train the machine such that it can 
provide a close approximation to the exact decision plane using far less 
number of training points and without loosing much of the generalization 
performance of the classical approach. The developed procedure closely 
preserves the accuracy of standard One-class nu-SVMs while reducing both 
training time and test time by several factors.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
NOAA-20 VIIRS Level-Regional Regional 11µm Day/Night Sea Surface Temperature (SST) Data, version R2024.0

The Visible and Infrared Imager/Radiometer Suite (VIIRS) is a multi-disciplinary instrument that is being flown on the Joint Polar Satellite System (JPSS) series of spacecraft, including the Suomi National Polar-orbiting Partnership (S-NPP) that launched in October 2011. JPSS is a multi-platform, multi-agency program that consolidates the polar orbiting spacecraft of NASA and the National Oceanic and Atmospheric Administration (NOAA). S-NPP is the initial spacecraft in this series, and VIIRS is the successor to MODIS for Earth science data product generation. VIIRS has 22 spectral bands ranging from 412 nm to 12 nm. There are 16 moderate-resolution bands (750m at nadir), 5 image-resolution bands (375m), and one day-night band (DNB).

Tags: earth-science, ocean-temperature, oceans
Organization: National Aeronautics and Space Administration
SMAP/SMOS L3 Radiometer 9 km EASE-Grid Soil Moisture V001

The Soil Moisture Active Passive (SMAP, launched in 2015) and the Soil Moisture and Ocean Salinity (SMOS, launched in 2009) missions are each L-band satellites that provide brightness temperature and soil moisture estimates and vegetation optical depth approximately every 2-3 days with a spatial resolution of ~40 km. By integrating brightness temperature observations from both satellites, these data products reduce the revisit time to about one day, improving the ability to monitor fast-response processes such as groundwater drainage and recharge, early dry-down after storms, and pre-storm soil moisture conditions for runoff determination. <br>
The Integrated SMAP and SMOS Soil Moisture Data is available in two products: <ul>
<li>SMOS-Based SMAP L2 Radiometer Half-Orbit 9 km EASE-Grid Soil Moisture (NSIDC-0799): This product is generated by processing half-orbit granules (either 6 AM ascending or 6 PM descending) from the pre-processed SMOS product. The L2 product consists solely of SMOS-based data, incorporating information after SMOS brightness temperatures (TBs) at a 40° incidence angle were inter-calibrated and the soil moisture (SM) and vegetation optical depth (VOD) were retrieved using the SMAP Dual Channel Algorithm (DCA). </li>
<li>SMAP/SMOS L3 Radiometer 9 km EASE-Grid Soil Moisture (NSIDC-0800): To generate this product, processing software ingests one day’s worth of granules from both the SMAP SPL2SMP_E product and the SMOS-derived L2 data. The L3 product contains both SMAP and SMOS descending (6:00 AM/PM) and ascending (6:00 PM/AM) data stored in separate arrays. </li>
</ul>

Tags: biosphere, earth-science, land-surface, microwave, soils, spectral-engineering, vegetation
Organization: National Aeronautics and Space Administration
TRMM PR Gridded Orbital Spectral Latent Heating Profiles L3 1.5 hours 0.5 degree x 0.5 degree V7 (TRMM_3G25) at GES DISC

3G25, "Gridded Orbital Spectral Latent Heating", produces 0.5 degree x 0.5 degree latent heating, Q1-QR, and Q2 profiles from Precipitation Radar (PR) rain. The granule size is one orbit.

Tags: atmosphere, clouds, earth-science
Organization: National Aeronautics and Space Administration
NASA 3D Models: Kepler

The Kepler Mission is specifically designed to survey a portion of our region of the Milky Way galaxy to discover Earth-sized planets in or near the habitable zones of their stars and determine how many of the billions of stars in our galaxy have such planets. Kepler was launched in March 2009.

Tags: 3d-model, kepler, satellite, spacecraft
Organization: National Aeronautics and Space Administration
Energy Analysis of Multi-Function Devices in an Office Environment

As part of an effort to monitor electricity usage by plug loads in a new high performance office building, plug load management devices were deployed to enable data collection, analysis, and active control of plug loads. We used a Commercial Off-The-Shelf (COTS) plug load management system to capture relevant data for two different types of multi-function devices (MFDs) in the facility, one of which was tested for use with different power settings. This enabled a quantitative analysis to assess impacts on energy consumption. It was found that a projected 65% reduction in annual energy consumption would result by using a newer, Energy Star compliant model of MFD, and an additional projected 39% reduction in annual energy consumption would result by subsequently changing the time-to-sleep for that MFD. It was also found that it may be beneficial to apply automated analysis with anomaly detection algorithms to detect problems with MFD performance, such as a failure to go to sleep mode or variations in sleep power draw. Furthermore, we observed that energy savings realized by using plug load management devices to de-energize (unplug) MFDs during non-business hours depends on the sleep power draw and time-to-sleep setting. For the MFDs in this study with settings established per the maintenance contract (which were different than factory default values), turning the device off at night and then on in the morning used more energy than leaving it on in sleep mode due to the start-up behavior and excessive time-to-sleep setting of four hours. From this and other assessments, we offer these recommendations to building occupants: reduce MFD time-to-sleep, encourage employees to use the power save button, and apply automated analysis to detect problems with device performance.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
Magnetically-coupled microcalorimeter arrays for x-ray astrophysics with sub-eV spectral resolution and large format capability Project

"We propose to develop a revolutionary x-ray camera for astrophysical imaging spectroscopy. High-resolution x-ray spectroscopy is a powerful tool for studying the evolving universe. Emission line ratios (e.g. within the He-like triplet) provide density and temperature diagnostics. Emission and absorption line energies identify ions and determine their velocities, and the shape of the lines can be used to study turbulence or the relativistic effects of a supermassive black hole. The grating spectrometers on the XMM and Chandra satellites demonstrated the power of high-resolution x-ray spectroscopy for astrophysics, but there remains a need for instrumentation that can provide higher spectral resolution with high throughput in the Fe-K band and that can enable spatial/spectral investigations of extended sources, such as supernova remnants and galaxy clusters. The instrumentation needed is a broad-band imaging spectrometer - basically an x-ray camera that can precisely resolve x-ray energies and fluxes over a large field-of-view.



 While we do not claim that in 3 years we will have developed such detectors, we advocate developing the technology that has the greatest potential for achieving this. Theoretically, magnetically-coupled microcalorimeters are best equipped to achieve sub-eV energy resolution in very large formats. We propose to build upon the work carried out by our group on metallic magnetic calorimeters (MMC) in the antecedent program. The great promise of MMCs for sub-eV energy resolution has been recognized for years. During our current research program, an accident in detector fabrication produced devices that derived their sensitivity from a different operating principle - the temperature dependence of a superconduc

Tags: active, project
Organization: National Aeronautics and Space Administration
High Mountain Asia COAWST Monthly 4km Regional Climate Model Simulations V001

This data product contains either monthly averaged or monthly accumulated modeled data in the High Mountain Asia region, generated by the Coupled-Ocean-Atmosphere-Waves-Sediment Transport (COAWST) modeling system (operated as a regional climate model). These modeled data span 15 years and have been used by the NASA High Mountain Asia Team (HiMAT) to research water resource use.

Tags: atmosphere, atmospheric-radiation, atmospheric-water-vapor, atmospheric-winds, cryosphere, earth-science, precipitation, snow-ice
Organization: National Aeronautics and Space Administration
SMAPVEX19-22 Millbrook Lidar Derived Digital Surface Model V001

These digital surface model (DSM) data consist of surface elevations derived from source lidar measurements collected in August 2022 in the vicinity of Millbrook, NY during the SMAPVEX19-22 campaign. The location was selected due to its forested land cover, as SMAPVEX19-22 aims to validate satellite derived soil moisture estimates in forested areas. The August collection period was selected to characterize ‘leaf-on’ conditions. DSM data represents the highest elevation of features on the Earth’s surface, which may include bare-earth, vegetation, and human-made objects.

Tags: earth-science, land-surface, topography
Organization: National Aeronautics and Space Administration
NEW RSW & Wall Fine Mixed Element Grid

RSW Fine Mixed Element Grid with viscous root wind tunnel wall.  This grid is for a node-based unstructured solver.

Quad Surface Faces=     38016
Tria Surface Faces=    477226
Nodes             =  23988100
Total Elements    =  58923530
Hex Elements      =         0
Pent_5 Elements   =     70279
Pent_6 Elements   =  41974373
Tet Elements      =  16878878
BL Tet Elements   =     69729

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration
A Local Scalable Distributed EM Algorithm for Large P2P Networks

his paper describes a local and distributed expectation maximization algorithm for learning parameters of Gaussian mixture models (GMM) in large peer-to-peer (P2P) environments. The algorithm can be used for a variety of well-known data mining tasks in distributed environments such as clustering, anomaly detection, target tracking, and density estimation to name a few, necessary for many emerging P2P applications in bioinformatics, webmining and sensor networks. Centralizing all or some of the data to build global models is impractical in such P2P environments because of the large number of data sources, the asynchronous nature of the P2P networks, and dynamic nature of the data/network. The proposed algorithm takes a two-step approach. In the monitoring phase, the algorithm checks if the model ‘quality’ is acceptable by using an efficient local algorithm. This is then used as a feedback loop to sample data from the network and rebuild the GMM when it is outdated. We present thorough experimental results to verify our theoretical claims.

Tags: ames, dashlink, nasa
Organization: National Aeronautics and Space Administration