DOI: 10.5281/zenodo.18340508
title: EMSeek Dataset

description: EMSeek Dataset Overview

Background

The EMSeek dataset accompanies the paper Bridging Electron Microscopy and Materials Analysis with an Autonomous Agentic Platform (Chen et al., 2025). It supports the system's reference-guided, single-click segmentation workflows and provides curated examples for downstream lattice reconstruction, property prediction, and literature-grounded reasoning. The collection spans atomic-resolution and nanoscale transmission electron microscopy scenarios so that SegMentor, EM2CIF, MatProphet, and related agents can be trained and evaluated on a unified benchmark of heterogeneous materials tasks.

Directory Layout

Task-specific data are organized at the repository root:

atom_columns/
atomic_defects/
metal_alloy_defects/
nanoparticles/
single_atom_catalysts/
README.md

Each task directory contains one folder per material or specimen. Inside every material folder the files follow a consistent structure:

task/material/
├── image/        # Raw microscopy frames (.png)
├── label/        # Pixel-wise masks (.png)
├── bbox/         # YOLO-style annotations (.txt)
└── prompt.txt    # Optional textual prompt (one-word / concise / detailed)

prompt.txt exists for all current materials except atomic_defects/WSe2.

Task Coverage

Atomic Column Segmentation (atom_columns)



Materials: DyScO3, LiCoO2, MoS2, NiO, Pt, Si, SrTiO3, graphene.

13,356 image/label/bbox triplets; all images are 256x256 grayscale.

Labels are stored as RGB PNGs but are binary (two colors).

Bounding boxes mark atomic-column centroids for detection.


Atomic Defect Identification (atomic_defects/WSe2)



107 grayscale images at 1024x1024 resolution.

RGB masks use three colors (background plus two defect-related classes).

Bounding boxes highlight each defect center; no prompt file in this task.


Metal Alloy Defects (metal_alloy_defects/FeCrAl)



182 images (174 at 1024x1024, 8 at 2048x2048) with matching bboxes.

183 grayscale labels (one extra label file without a matching image).

Prompts summarize defect characteristics for language-aware agents.


Nanoparticle Analysis (nanoparticles)



Materials: Au@C, Au@Ge, Au@SiN, Au@zeolite, CdSe@C, Fe3O4, Fe3O4@SiO2, Pd@C, PtSn@Al2O3.

822 images; 821 labels/bboxes (one missing label/bbox for Au@C).

Image sizes vary from 1002x668 up to 4096x4096; 1024x1024 and 2048x2048 are common.

Labels are mostly binary grayscale PNGs, with a small number stored as RGB/RGBA.


Single-Atom Catalysts (single_atom_catalysts/Pt@NC)



25 grayscale images (512x512, 1024x1024, and 2048x2048).

RGB masks are binary (two colors), with matching bboxes and prompts.


Annotation Formats



Images (image/*.png): PNG microscopy frames. Most tasks use single-channel grayscale; the nanoparticle set includes both grayscale and RGB images. Atomic column data are 256x256, while other tasks keep native resolutions listed above.

Masks (label/*.png): PNG masks stored as either grayscale or RGB. Atomic defects use three RGB colors, while atom columns and Pt@NC are binary masks stored in RGB format. Nanoparticle masks are mostly binary grayscale with a few RGB/RGBA files.

Bounding boxes (bbox/*.txt): Provided in normalized YOLO format as <class> <x_center> <y_center> <width> <height>. Coordinates and extents are expressed between 0 and 1.

Prompts (prompt.txt): Contain one-word, concise, and detailed textual descriptions for language-conditioned workflows. Prompts are absent in atomic_defects/WSe2.


Usage Notes



Use image/ and label/ pairs to train segmentation models; bbox/ Files can drive detection or assist with weakly supervised learning.

For integration with the EMSeek pipeline, pass segmentation masks to EM2CIF for mask-aware lattice reconstruction and then to MatProphet for property inference, recreating the paper's pixels-to-properties loop.

When handling large nanoparticle images, consider tile-based sampling, multi-scale augmentations, or down-sampling to fit GPU memory constraints.

A few directories include system-generated files (for example ._*.png); filter them out during preprocessing to avoid loading errors.


Citation

When using this dataset, please cite the original work:

Chen, G., Yuan, W., & You, F. (2025). Bridging Electron Microscopy and Materials Analysis with an Autonomous Agentic Platform. Cornell University AI for Science Institute.