AndreaBozzo/ceres-open-data-index huggingface dataset:
PARETO 0.3.0 Release

PARETO 0.3.0 Release. Highlights:

- Fixing Warning in get_data
- Clean up of old main function which has been replaced by the run files
- Adding plot_scatter() function
- Support node capacity
- Add Gurobi troubleshooting files to gitignore by @melody-shellman
- Set slacks to zero
- Water treatment case study
- Fix defaultvalue treatement
- Fix bug for incorrect number of input arguments in water quality
- Pipeline configs documentation
- Enable rendering Unicode characters in ReadTheDocs PDF builds
- Adding support for units
- Add discrete water quality to the model
- Updating documentation for strategic model, and corresponding fixes to strategic model
- Rework getting started docs build section and more
- Enable Codecov reports in CI builds
- Correct error in CompletionsPadSupplyBalanceRule

Tags: PARETO, PSE, Process Systems Engineering, Produced water, Water
Offshore Geologic Carbon Storage (GCS) Inventory

The Offshore Geological Carbon Storage Inventory Web Map is an online web mapping application designed to help users explore and visualize the Offshore Geologic Carbon Storage Inventory. This dataset is an inventory of offshore geologic carbon storage (GCS) projects and studies, gathered to summarize ongoing GCS efforts taking place offshore globally. This inventory includes both actualized projects as well as characterization studies and aggregates a variety of attribute fields for each project / study. It is intended to be used for research and comparison purposes, see full disclaimer and credits.

Tags: Assessment, CCS, CO2, Carbon, Carbon Dioxide, Carbon Storage, Characterization, Development, GCS, Geologic Carbon Storage, Geologic Storage, Geology, Global, Injection, International, Inventory, Monitoring, Offshore, Offshore Carbon Storage, Potential, Project Progression, Project Regulation, Regulation, Reservoir, Saline, Saline Aquifer, Sequestration, United States
Offshore Geologic Carbon Storage (GCS) Data Collection Web Application

The Offshore Geologic Carbon Storage Data Collection Web Application is an interactive data collection which aggregates and disseminates publicly available data to support offshore geologic carbon storage (GCS) in the United States. This data collection can be leveraged by stakeholders to understand where GCS may be viable offshore, create GCS project analogs, and address challenges to GCS in offshore environments. Use of this tool is solely at the discretion of the user. See full Federal Disclaimer for further information (https://netl.doe.gov/home/disclaimer).

Tags: Carbon Sequestration, Geologic data, carbon storage, offshore, web application
Electricity Baseline 2022 Background Data and Log File

The ElectricityLCI v2 Python package (https://github.com/USEPA/ElectricityLCI/tree/v2.0) was used to generate the 2022 electricity baseline: a regionalized life cycle inventory model of U.S. electricity generation, consumption, and distribution using standardized facility and generation data.

ElectricityLCI implements a local data store for downloading and accessing public data on an individual's computer. The data store follows the folder definition provided by USEPA's esupy Python package (https://github.com/USEPA/esupy), which utilized the appdirs Python dependency (https://pypi.org/project/appdirs/).

This submission includes the background data used to generate the 2022 electricity baseline inventory. Each zip archive stores the source files as found in their data stores. Sub-folders in each of the data stores are archived separately. For example, stewi.zip contains the JSON files, while stewi.facility.zip is the 'facility' sub-folder of stewi data store that stores the parquet files.

To reproduce the data store, extract each zip file and drag-and-drop sub-folders in to their appropriate root folders to recreate the data stores, then copy the root folders to your data store folder (as returned by running the following on the command line: `python -c "import appdirs; print(appdirs.user_data_dir())"`). The main five data stores include: 'electricitylci', 'facilitymatcher', 'fedelemflowlist', 'stewi', and 'stewicombo'.

The log file generated by the 2022 model run is also included, which contains the statements at the DEBUG level and above.

Tags: Electricity, LCA, data inventory
National Minerals Information Center Commodity Statistics and Information

Statistics and information on the worldwide supply of, demand for, and flow of minerals and materials essential to the U.S. economy, the national security, and protection of the environment.

Respondents with questions about the canvass forms can contact the Mineral Commodities Data Unit, U.S. Geological Survey, 985 National Center, Reston, VA, 20192. Telephone: (703) 648-7960. Email: nmic_canvass_forms@usgs.gov.

Tags: Critical Minerals, Mineralogy, USGS, United States Geological Survey, mineral commodity, mineral resources
Harden Core Hole C-CN-3 (Nowata County, OK) Electron Microprobe Analysis (Oklahoma Geological Survey)

The Kansas Geological Survey received the attached electron microprobe documentation as a deliverable for the CoreCM in Coaly Strata located in the Cherokee-Forest City Basin project (DE-FE0032056). This submitted research was conducted by the Oklahoma Geological Survey and is being submitted as it was received without modification.  Data contained within references 2 samples from the Harden Core Hole (C-CN-3) well located in Nowata County, Oklahoma.

Tags: Electron Microprobe, Hardin Core Hole C-CN-3, Nowata County, OPIC, Oklahoma, Oklahoma Geological Survey
CT Scanning and Geophysical Measurements of the J.F. Ware 1-9 Well in Smith County, Texas

Report describing the data of the Cotton Valley Group from a well in Smith County, Texas. Data is available on EDX here:
Kylie P Brinza, Thomas Paronish, Dustin Crandall, Shelby Isom, Terry McKisic, WARE 1-9 Well, 1/8/2025, https://edx.netl.doe.gov/dataset/ware-1-9, DOI: 10.18141/2477580

And the report citation is:
Brinza, K.; Paronish, T.; Crandall, D.; Isom, S. L.; McKisic, T.; Jarvis, K.; Dotson, B. Computed Tomography Scanning and Geophysical Measurements of the J.F. Ware Gas Unit 1-9 Well in Smith County, Texas; DOE/NETL-2025/4918; NETL Technical Report Series; U.S. Department of Energy, National Energy Technology Laboratory: Morgantown, WV, 2025; p 94. DOI: https://doi.org/10.2172/2530610 

Tags: Core Characterization, Critical Minerals
Subsurface Energy Systems Mapping Inquiry Tool (MapIT)

The Subsurface Energy Systems Mapping Inquiry Tool (MapIT) is an online web mapping tool designed to help users discover available public-sourced data to facilitate data exploration for subsurface energy exploration and characterization efforts for resource identification (e.g. critical minerals, hydrocarbons, geothermal) as well as injection of geologic sequestration of carbon dioxide (e.g. enhanced oil recovery, saline storage, etc.). Modules within the tool curate data related to geology, faults, fractures, injection and confining zones, hydrologic information, groundwater, groundwater wells, geomechanical and petrophysical data, and geochemical data. 

User documentation on how to use the tool is also provided. 

Data have been collected from authoritative national, state, and local sources and made available in this tool. The data is also available as a data catalog and Esri Geodatabase at:  https://edx.netl.doe.gov/dataset/mapit-database

Disclaimer:

There is no guarantee of completeness or appropriateness for individual user’s requirements. Use of this tool is solely at the discretion of the user. See full Federal Disclaimer for further information (https://netl.doe.gov/home/disclaimer).

This project was funded by the United States Department of Energy, National Energy Technology Laboratory, in part, through a site support contract. Neither the United States Government nor any agency thereof, nor any of their employees, nor the support contractor, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights.  Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.  https://www.netl.doe.gov/home/disclaimer

Tags: Carbon Sequestration, Class VI, Class VI Data Support Tool, EDX Spatial, EOR, Resource Assessment, Spatial data, carbon storage, geologic carbon storage, subsurface data
Geoanalytical Evaluation of Saline Storage (GEESS) Geodatabase v2.0

The Geoanalytical Economic Evaluation of Saline Storage (GEESS) geodatabase was developed to support the United States Department of Energy (DOE) and National Energy Technology Laboratory (NETL) in their geologic carbon storage efforts by characterizing saline geologic formations present in the FECM/NETL CO2 Saline Storage Cost Model (CO2_S_COM) [1]. Using publicly available literature and data, the GEESS geodatabase characterizes 57 geologic formations across the lower-48 U.S. states in what are called Fully Integrated Geodatabases (FIGs). The FIG is a vector polygon feature containing thousands or tens of thousands of individual polygons, which each contain discrete geologic parameter values. A list of the critical geologic parameters that are characterized in the GEESS geodatabase are described in the “Processing Steps and Workflow” part of the ReadMe file, as well as the Data Catalog accompanying the GEESS geodatabase.

The FIG is the basis of the GEESS geodatabase and is the direct representation of the collected geologic data. In addition to the FIG, the GEESS system contains grid files. Due to the complexity of the FIGs, grids are used to sample the geologic data so they can be exercised within CO2_S_COM. The grid files contain the geologic data sampled from the FIG, as well as estimates of “Plume Uncertainty Diameter” and “First-year Break-even Price of CO2” derived from CO2_S_COM based on the GEESS grid data. 

Tags: CO2 plume, GEESS, Geoanalytical Economic Evaluation of Saline Storage, Geologic Sequestrion, Lithology, Saline Storage, Thickness, United States, carbon, depositional environment, depth, first-year break-even, fracture pressure, geoanalytical, geodatabase, geologic, geologic carbon storage, geology, plume diameter, porosity, pressure, salinity, technoeconomic, temperature
Projected Operational Energy Life Cycle Data 2025

These excel files contain the environmental emission projections for key building energy sources: electricity, natural gas, propane (LPG), and fuel oil. The sources and methods are provided in the report "Projected Operational Energy Life Cycle Data Development: 2025 Update." The report and data are an update to the data previously posted here: https://netl.doe.gov/energy-analysis/details?id=f8890fac-be55-44ac-aaa9-e2888bfabe93

Tags: Natural Gas, Propane, TRACI, annual energy outlook, cambium, fuel oil
2020 natural gas LCA appendices Rev1

This collection is the data-centric appendices for the report Life Cycle Analysis of Natural Gas Extraction and Power Generation: U.S. 2020 Emissions Profile. It consists of Appendix A: Additional Modeling Parameters [spreadsheet]; Appendix B: Water Burdens [spreadsheet]; Appendix D: Simulation of Liquids Unloading [python script and spreadsheet]; Appendix E: Detailed GHG Results for All Scenarios [spreadsheet]; Appendix F: Full Inventory Results [spreadsheet]; and Appendix I: Stage-Level Natural Gas Loss and Consumption Rates [spreadsheet].

These results have been updated from the previous version (https://edx.netl.doe.gov/dataset/2020-natural-gas-lca-data-appendices) to correct a modeling error where the same post-processing natural gas composition was used instead of the intended regional compositions.

Tags: Appendices, GWP, LCA, LCI, Natural Gas
NETL Natural Gas Lifecycle Model 2020 Rev1

This is the excel-based life cycle model that contains all the parameters and Monte Carlo simulation capabilities to model the techno-regions contained in the report: Life Cycle Analysis of Natural Gas Extraction and Power Generation: U.S. 2020 Emissions Profile. This is an update to a previous version (https://edx.netl.doe.gov/dataset/netl-natural-gas-lifecycle-model) and corrects a modeling error in the post-processing natural gas composition.

Tags: Calcium Chloride, LCA, Monte Carlo simulation, Natural Gas, Shale gas, conventional gas, life cycle analysis, life cycle assessment, unconventional gas
NETL RDE Image Classification Dataset 2020 - 10 Classes

Dataset including high-speed down-axis RDE images used for updated image classification study. This dataset includes 100,000 images with 10 classifications: 1CW, 1CCW, 2CW, 2CCW, 3CW, 3CCW, and Deflagration. Images are cropped to center annulus, and resized to 301x301 pixels. Images are filtered using the AFRL Beta correction factor.

Tags: Dataset, Machine Learning, Rotating Detonation Engine
NETL RDE Image Classification Dataset 2025 - 14 Classes

Dataset including high-speed down-axis RDE images used for updated image classification study.  This dataset includes 180,000 images with 14 classifications: 1CW, 1CCW, 2CW, 2CCW, 3CW, 3CCW, Deflagration, 4CW, 4CCW, 5CW, and 5CCW.  Images are cropped to center annulus, and resized to 301x301 pixels. Images are filtered using the AFRL Beta correction factor.

Tags: Dataset, Machine Learning, Rotating Detonation Engine
Mapping Inquiry Tool (MapIT) Database

The Mapping Inquiry Tool (MapIT) database consists of a geodatabase and data catalog of geologic, geophysical, structural, hydrologic, and contextual data, based on the data types to support geologic carbon storage activities and other subsurface energy systems resource assessments. The database was aggregated from publicly available data across the USA from state and federal entities. The database is structured by categories including rock unit geology, boundaries, national CS datasets, geophysical data, faults and structural data, infrastructure, surface hydrology, groundwater, and more.

The data described in the data catalog is also available in the Mapping Inquiry Tool (https://edx.netl.doe.gov/dataset/mapping-inquiry-tool).

Version 3 of the geodatabase and data catalog have been updated as of 5/17/2024. The database was published with a limited number of layers. The Catalog V3 contains many more resources than the geodatabase, documenting all layers that will be included in MapIT, and includes links to the original sources of the data. Within the catalog, in the final column, there is information about if the file is included in the geodatabase or not. Use the links provided in the catalog to download data directly from the original source if not included in the geodatabase.

Four resources are included in this submission:

1. Geodatabase

2. ReadMe file 

3. Catalog of data layers and additional data resources 

4. Web link to a resource describing the motivation and reviewing the content of the geodatabase - DOE NETL Carbon Storage Site Mapping Inquiry Tool Database

Tags: Class VI, Class VI Data Support Tool, Class VI Data Support Tool Geodatabase, DisCO2ver, EDX Spatial, EOR, Geologic data, Hydologic data, Resource Assessments, carbon storage, geodatabase, geologic carbon storage, geophysical data, subsurface data, surface waters, wetlands
Computed Tomography Scanning and  Geophysical Measurements of the  Clinton Formation in Ohio

Report describing the data of the Clinton Formation in Ohio. Data is available on EDX here: Thomas Paronish,  Andrew James Holleran, Mathias Pohl, Dustin Crandall, Karl Jarvis, Scott Workman, Jessica Drosche, Terry McKisic, Catherine Collins, Mindy Thomas, James McDonald, CT and Geophysical Data of Clinton Sandstone Cores from Ohio, 9/2/2025, https://edx.netl.doe.gov/dataset/ct-and-geophysical-data-of-ohio-clinton-sands DOI: 10.18141/2587096

And the report citation is: 
 Paronish, T.; Holleran, A.; Pohl, M.; Crandall, D.; Jarvis, K.; Workman, S; Drosche, J.; McKisic, T.; Collins, C.; Thomas, M.; McDonald, J. Computed Tomography Scanning and Geophysical Measurements of the Clinton Formation in Ohio; DOE.NETL-2025.4949; NETL Technical Report Series; U.S. Department of Energy, National Energy Technology Laboratory: Morgantown, WV, 2025; p 68.


Tags: Computed Tomography, Core Characterization, Core Logging, Ohio, critical mineral, sandstone
CT and Geophysical Data of Clinton Sandstone Cores from Ohio

Collection of computed tomography and multi-sensor core logger data of 12 wells in Ohio that intersect the Clinton Sandstone formation. This data is described along with well information in a technical report series document:
Paronish, T.; Holleran, A.; Pohl, M.; Crandall, D.; Jarvis, K.; Workman, S.; Drosche, J.; McKisic, T.; Collins, C.; Thomas, M.; McDonald, J. Computed Tomography Scanning and Geophysical Measurements of the Clinton Sandstone in Ohio; DOE.NETL-2025.4949; NETL Technical Report Series; U.S. Department of Energy, National Energy Technology Laboratory, Morgantown, WV, 2025; p 68. https://doi.org/10.2172/2589262

Tags: Clinton Sandstone, Core Characterization, Ohio, carbon storage
CCSI Toolset 3.17 Release

CCSI Toolset 3.17 Release Highlights

A workaround was developed to allow complex Aspen Custom Modeler (ACM) models to be used in FOQUS. This workaround uses Visual Basic for Applications to connect the ACM models to FOQUS.

The ability for User plugins to be uploaded to FOQUS Cloud was added. 

The documentation was updated to include Optional Software Install and Tutorial Notes to clarify the usage of Turbine and SimSinter in installation instructions and adds a link to the relevant tutorial page. The Sequential Design of Experiments documentation was updated with current screenshots. 

The copyright was updated to include 2023. 


Tags: FOQUS, Process Systems Engineering
CCSI Toolset 3.18 Release

CCSI Toolset 3.18 Release Highlights

FOQUS was updated to allow installation for users using MacOS on Apple silicon.

FOQUS Cloud support was added for user plugins. 

The Optimality-Based Design of Experiments tutorials were updated to reflect the latest changes in the user interface flow. 

The plot discrete sliders were fixed for CDF and 3D plots within Uncertainty Quantification, which were not working due to a matplotlib depreciation.

The installation was updated to set the default location for the PSUADE executable if found in the environment. 

Updates were made to allow compatibility with NumPy 1.25.

Additional documentation changes were made to fix typographical errors and fix a broken link to optional software.


Tags: FOQUS, Process Systems Engineering
CCSI Toolset 3.19 Release

CCSI Toolset 3.19 Release Highlights

A gradient generation tool was developed to support GENN models in FOQUS. Certain machine learning tools train gradient-enhanced neural network (GENN) models which can be more accurate for complex datasets given a priori knowledge of model derivatives. However, the derivatives must be known beforehand and are not often available for process data. This tool automatically predicts the gradients for a training dataset in a form usable by common GENN trainers, such as Surrogate Modeling Toolbox.

Support was added for Surrogate Modeling Toolbox GENN models in FOQUS, including updates to the run methods, node properties, test framework, documentation and optional dependencies list. Users can train/save Surrogate Modeling Toolbox gradient-enhanced neural network (GENN) models with custom objects and produce .pkl files compatible with the Machine Learning/Artificial Intelligence Plugin in FOQUS. 

A simpler implementation of the ordering algorithm in the Sequential Design of Experiments (SDOE) module was included. The SDOE examples documentation was updated. 

The Optimality-Based Design of Experiments was updated to improve the error handling when the results are None.


Tags: FOQUS, Process Systems Engineering
CCSI Toolset 3.20 Release

CCSI Toolset 3.20 Release Highlights

Minimum Viable Product surrogate plugin was added for creating Machine Learning/Artificial Intelligence models. Corresponding documentation was added for the plugin.

Sequential Design of Experiments plots were updated to eliminate an issue with the window stack ordering upon closure of the plots.

Support for Python 3.7 was removed. Documentation was improved by adding new mandatory section to the ReadTheDocs configuration and adding installation instructions back for NLOpt.

TurbineLite was updated to 3.0.0, which is compatible with SimSinter 3.0.0. The developer environment was updated and 32-bit support was removed.

SimSinter was updated to 3.0.0. This version removed gPROMS support and included security updates.


Tags: FOQUS, Process Systems Engineering, SimSinter, TurbineLite
CCSI Toolset 3.21 Release

CCSI Toolset 3.21 Release Highlights

Parallelization support was added for Sequential Design of Experiments (SDOE) computations using Dask (preliminary). Input type dependent ordering capability was added to the SDOE module. With this implementation the user can specify the level of difficulty to change an input (Easy or Hard) and FOQUS will generate the appropriate ordered design depending on the input difficulty combination.

Python version support was extended. FOQUS is now compatible with Python 3.8 through 3.12.

Platforms used for automated testing were expanded to include macOS ARM (Apple Silicon).

Updates to the FOQUS documentation to include information on how to set paths for SimSinter and TurbineLite. Turbine configuration section was added to Debugging Documentation.


Tags: FOQUS, Process Systems Engineering
CCSI Toolset 3.22 Release

CCSI Toolset 3.22 Release Highlights

The Sequential Design of Experiments user interface was updated to resolve an issue where the results would fail to plot in some cases (e.g., Non-Uniform Space Filling designs). 

The Machine Learning/Artificial Intelligence module was updated to support Keras 3 and to reflect changes made to dependencies’ syntax.

A check was added to ensure PSUADE is installed and available at FOQUS startup. If PSUADE is not installed, a link to the FOQUS documentation is displayed and FOQUS is closed.

The copyright year was updated to include 2024 in places where it had not previously been updated. Typographical errors were corrected to improve clarity in variable names and documentation. The FOQUS documentation was updated to reflect the fact that ALAMO can have two executables and indicates the correct executable to add to the Settings path.

SimSinter was updated to version 3.1.0. This version removed gPROMS support and included security updates.

Tags: FOQUS, Process Systems Engineering
CCSI Toolset 3.25 Release

CCSI Toolset 3.25 Release Highlights

The copyright in the CCSI Toolset was updated to include the year 2025. The code and notes within it were revised to correct minor typographical errors and clarify meanings of variables. The configuration of the FOQUS documentation via Read the Docs was updated to explicitly set the path.


Tags: FOQUS, Process Systems Engineering
United States CMM Insights Dataset

Geo-data science applications for critical mineral analysis, development acceleration, economic impact assessment, and project efficiency.

Coverage: 10 years (2014-2023), 3 geographic levels (county, state, tract), 45 features

Data Categories:
- census: 30 features (e.g., asian population percentage, black population percentage, citizen voting age population percentage)
- ejscreen: 3 features (e.g., P_DSLPM, P_PM25, P_PWDIS)
- energyexpenditure: 12 features (e.g., Housing adjustment factor, Income adjustment factor, Monthly electricity cost)


Tags: ClaiMM, Critical Minerals and Materials, DOE, Department of Energy, Energy Community, NETL, National Energy Technology Laboratory