metadata

license: openrail
license_name: open-rail-m
license_link: https://raw.githubusercontent.com/Clay-foundation/model/main/LICENSE-MODEL.md

Clay documentation

Overview

Clay is a foundational model of Earth using Earth Observation data. As the AI Deep Learning architecture, it uses an expanded visual transformer upgraded to understant geospatial and temporal relations on Earth data, from any instrument/spectral data. The AI self-supervised fundational task is a Masked Autoencoder (MAE) approach for training.

The Clay model primarily functions in two ways: first, by directly generating semantic embeddings for tasks like similarity searches, and second, through fine-tuning its outputs with additional data labels. This fine-tuning supports various tasks, including classification (e.g. flood detection and deforestation monitoring), regression (e.g. estimating carbon stock or crop yields), and generative tasks such as creating RGB imagery from SAR data. Moreover, users can further enhance model performance by incorporating higher-resolution data.

This documentation uses nbdev, which combines documentation, code samples and an SDK. This means that every page is also a python notebook anyone can use, with practical code examples for each functionality, and use case. Moreover, you can install pip install clay and use the same functions.

Clay is open source, open data and open for business.

Where is what

Our website is madewithclay.org.
The Clay model code lives on Github. License: Apache.
The Clay model weights live on Huggin Face. License: OpenRAIL-M.
The Clay documentation lives on this site. License: CC-BY.
The Clay SDK lives on PyPi. License: Apache.
We maintain a set of embeddings on Source Cooperative. License: ODC-BY.

How to use Clay

The model can be used in two main ways:

Directly, use it to make inference. See Model
1. Check and run Benchmarks on the model. See Benchmarks
Generating semantic embeddings. E.g. for Similarity search. See Embeddings.
Fine-tunning the model for other tasks, or for other input data. E.g. flood detection, crop yields, … See Fine-tunning.

How to contribute

Clay is an open source project, and we welcome contributions of all kinds.

The Documentation, python package and notebooks are all the same NBdev project, located here.

Note: If you want to contribute to the model code, please check the model repository.

To install the nbdev project locally, you can use:

git clone git@github.com:Clay-foundation/documentation.git
cd documentation
pip install nbdev
nbdev_install_git_hooks

After you make changes, you can export the notebooks into both the package, rendered documentation and clean jupyter notebook execution metadata with:

nbdev_prepare

If you want to preview the documentation locally, you can use:

nbdev_preview

To run the test locally, you need to install Github CLI and act extension sudo gh extension install nektos/gh-act.

The “Clay model releases” folder uses a lot of resources to document the version releases. To run these you also need access to the S3 bucket with outputs and all the embeddgins. You will need a local file (e.g. .secrets) with the AWS credentials to read the Clay buckets. Remember to confirm this file is on .gitignore to avoid commiting it.

Then you can run the tests with:

gh act --secret-file .secrets

–

Clay is a fiscally sponsored project of Radiant Earth, a USA registered 501(c)3 non-profit.