OneScience-Group/ClayFoundation

Clay Foundation Model

Model Introduction

Clay Foundation Model is an independent engineering reproduction of the public Clay v1.5 specification. It jointly encodes satellite imagery from different sensors, band center wavelengths, ground sample distance, time, and latitude/longitude into Earth observation embeddings, and reconstructs input bands with a Masked Autoencoder. It can serve as a representation backbone for land-cover classification, regression, change detection, and other downstream remote-sensing tasks.

Official documentation: Clay Foundation Model v1.5
https://clay-foundation.github.io/model/release-notes/specification.html

Model Description

Clay Foundation Model was proposed by the Clay Foundation. The model was trained on multi-sensor Earth observation data including Sentinel-2, Landsat, Sentinel-1, NAIP, LINZ, and MODIS. It is suitable for remote-sensing image representation, multispectral reconstruction, land-cover classification, regression, change detection, and related tasks.

Use Cases

Use Case	Description
Multi-sensor representation	Use one model to process imagery with different numbers of bands from sensors such as Sentinel-2, Landsat, and Sentinel-1.
Dynamic spectral encoding	Dynamically generate Patch Embeddings from the center wavelengths of input bands to validate integration methods for new sensors.
Remote-sensing image reconstruction	Use a Masked Autoencoder to reconstruct masked patches in multispectral imagery.
Local engineering validation	Use a small amount of synthetic data to validate training, inference, evaluation, visualization, and checkpoint workflows.
Multi-GPU training	Launch distributed data-parallel training with torchrun.

Usage Instructions

1.OneCode

Experience intelligent, one-click AI4S programming through the OneCode online environment:

Try intelligent, one-click AI4S programming

2.Download and Installation

hf download OneScience-Group/ClayFoundation --local-dir ./ClayFoundation
cd ClayFoundation

Environment Dependencies

Hardware Requirements

• A GPU or DCU is recommended.
• A CPU can be used to verify connectivity with the default small-sample configuration; training the official-size model requires large-scale acceleration resources.
• DCU users must install DTK in advance. DTK 25.04.2 or later, or the OneScience-recommended version matching the current cluster, is recommended.

DCU Environment

# Activate DTK and CONDA first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# uv installation is supported
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai

GPU Environment

# Activate CONDA first
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
conda activate onescience311
# uv installation is supported
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai

Training Data

This repository uses a small number of synthetic samples to validate the engineering workflow. The training and test data are stored in data/train.npz and data/test.npz, respectively. The synthetic data contains 10-band Sentinel-2, 6-band Landsat, and 2-band Sentinel-1 imagery, with center wavelengths and ground sample distances corresponding to the official metadata for each sensor. Each sample also includes week, hour, latitude, and longitude metadata; a synthetic teacher representation target; and classification and regression targets for lightweight evaluation.

The default input size is 64×64, with four training samples and two test samples. This scale is intended only to validate the dynamic-band interface, spatiotemporal metadata encoding, MAE training, and multi-sensor inference workflow. It does not represent the data distribution or training scale of the approximately 70 million global remote-sensing chips used by the official model.

python scripts/fake_data.py

Training

python scripts/train.py

For multi-GPU training, use:

torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py

The default configuration is intended for rapid workflow validation. Formal experiments should use real remote-sensing data, the official-size configuration, a DINOv2 teacher, and a complete training schedule.

result/checkpoints/clayfoundation.pt
result/training/metrics.json

Trained Weights

This repository does not include synthetic or official weights in weight/. The Clay Foundation has released the official Clay v1.5 checkpoint:

https://huggingface.co/made-with-clay/Clay/resolve/main/v1.5/clay-v1.5.ckpt

This repository is a scaled-down independent engineering implementation. Its model parameter names and dimensions are incompatible with the official Encoder weights of approximately 1.25 GB. To use the official weights, use the model implementation and data preprocessing workflow provided by the official Clay repository.

Inference

python scripts/inference.py

Inference loads the training checkpoint and separately processes Sentinel-2, Landsat, and Sentinel-1 test imagery. It generates Encoder embeddings, L2-normalized projected embeddings, complete multispectral MAE reconstructions, reconstruction loss, and teacher representation alignment loss, and saves them to:

result/output/predictions.npz

Evaluation and Visualization

python scripts/result.py

The evaluation measures image reconstruction and representation quality across sensors and reports reconstruction error, embedding norms, and representation similarity. It also generates comparison plots of the input imagery, reconstructed results, and absolute errors. Results on synthetic data are intended only to validate the engineering workflow and do not represent the official Clay v1.5 training loss, real downstream-task performance, or cross-region generalization capability.

result/evaluation/metrics.json
result/evaluation/comparison.png

Official OneScience Information

Platform	OneScience Main Repository	Skills Repository
Gitee	https://gitee.com/onescience-ai/onescience	https://gitee.com/onescience-ai/oneskills
GitHub	https://github.com/onescience-ai/OneScience	https://github.com/onescience-ai/oneskills

Citation and License

This repository is an independent engineering reproduction of the public Clay Foundation Model v1.5 specification. The official Clay source code and model weights are licensed under Apache-2.0.

Official Clay repository:
https://github.com/Clay-foundation/model

Use of this repository's code, the official model weights, and the data remains subject to the licenses and terms of use of their respective projects.