CEOS Systems Engineering Tools
and Applications
GOFC and SilvaCarbon Workshops
Lima, Peru

Presented by: Paul Kessler
CEOS Systems Engineering OfficeNASA Langley Research Center
July 6, 2016

What is CEOS?
www.ceos.org

The Committee on Earth Observation Satellites (CEOS)
serves as a focal point for international coordination anddata exchange to optimize societal benefit from spacebased Earth observations. CEOS represents 22 countries
through its 31 space agencies and 28 associate members.

What is CEOS doing?
CEOS agencies collaborate on a variety of international
projects that address key global issues ...
Agriculture ... Global Food Security (GEOGLAM, UN-FAO)
Deforestation ... Carbon Stocks (GFOI, UN-REDD)
Climate Change ... Global Climate Change (GCOS, UNFCCC)
Disaster Management ... Floods, Earthquakes (UNISDR)
Data Access ... Promoting and improving space data use

CEOS Organization

What is COVE ?
 The CEOS Visualization Environment (COVE) is abrowser-based suite of tools for searching,analyzing and visualizing actual and potential
satellite sensor coverage.
 COVE is FREE and OPEN for anyone to use!
 COVE includes 260 missions, 705 missioninstrument combinations.
 COVE is linked to several mission archives to get
metadata and browse images for past acquired
data: Landsat, SPOT, Pleaides, Radarsat-2, ALOS-1,TerraSAR-X and Sentinel-1.
 There is a large international user base ...
5800+ unique users in 2015.
 The NASA SEO maintains COVE is constantly
assessing inquiries from the user community to add
new missions, instruments, mode and features tothe tool for expanded capability.

The COVE suite of tools includes:
 COVE – The main tool for globalvisualizations
 Rapid Acquisition Tool – A
tabular tool for analyses
 Coverage Analyzer - A tool for
long-term coverage analysis
 Mission and Instrument
Browser – Details on the COVE
mission database

Recent COVE Updates
 January 2016: Changed from Google
Earth to Cesium for globe interface
 New Missions: Sentinel-1A/2A,
CBERS-4, RCM 1-3 (notional), FY-3C,PROBA-V, TET-1.
 New Overlays: Landsat/Sentinel
constellation revisit performance,
Global Phenology (2001/2014
monthly NDVI Min/Max) New Analysis Tools: Custom Mission
Tool allows the creation of a notional
mission for analyses.
 New Archive Link: Sentinel-1A
 Future Archive Link: Sentinel-2A
Coming soon ...

Example of COVE connection to theSentinel-1A data archive. COVE now
shows actual acquired data locations,quick-look images, and links to order thedata.

SPOT and Pleaides in COVE
January 1-8, 2014 over Africa

Coverage Analyzer• Count historic acquisitions
over defined regions.
• Cloud filtering and custom
discritization possible.
• Missions: Landsat-7/8,SPOT-5/6, Pleaides-1A/1B,
Radarsat-2, Sentinel-1A.
• Output on Cesium globe ortable format.
• KML and CSV export.

Landsat-8 over Kenya
Year 2014, <20% cloudyData Policy Portal
www.ceos-datapolicy.org

Open Data Access = 71% of active CEOS missions
 CEOS Agencies are currently operating 135 missions
 Database includes 352 missions launched since 1990 and651 mission-instrument combinations.
 Tool includes direct links to primary data portals.
Landsat Country Reports• The CEOS Systems Engineering Office (SEO)
has prepared detailed historic Landsat
coverage reports for 70 GFOI countries.
• The SEO created these reports using
automated scripts connected to the Landsat
archive. All Landsat scenes (missions 5,7,8)
from 1990 through 2015 were included.
• These reports (PDF and EXCEL format) are
here: http://www.gfoi.org/space-data/• The reports include summary graphs, tables,and other detailed data for every acquired
scene (mission, path/row, cloud%, processinglevel).
• These reports will be valuable for countries toassess available scenes and cloud cover for
future data ordering.
• Updates will be completed annually at the endof the calendar year. Future plans include
adding Sentinel-1 and Sentinel-2 to thereports.

The Data ProblemLand Imaging Data Growth over Kenya
6.00

5.00

Data (TB)

4.00• Increased Volume• Low Capacity• Slow Internet

3.00

2.00

1.00

0.00

Year

• A significant growth in land imagery
data (e.g. Landsat, Sentinel) will
increase data volumes by >10x in
the next few years.
• Many countries lack theknowledge, infrastructure, andresources to access and use spacebased data.
• Countries have requested support
from CEOS for data access, storage,processing, and analysis.
• The new CEOS Data Cubearchitecture provides a solutionthat saves countries time andmoney and reduces technicalcomplexity.

What are Data Cubes?
• Data Cube = Time-series multi-dimensional(space, time, data type) stack of spatiallyaligned pixels ready for analysis• Proven concept by Geoscience Australia (GA)and the Australian Space Agency (CSIRO) andplanned for the future USGS Landsat archive.
• Shift in Paradigm ... Pixels vs Scenes• Analysis Ready Data (ARD) ... Dependent on
processed products to reduce processing
burden on users• Supports a multitude of applications,
reduces data preparation time, allows timeseries analyses, increases interoperability ofmultiple datasets.
• Open source software approach allows freeaccess, promotes expanded capabilities, andincreases data usage.

Open Source Software
https://github.com/data-cube

TIME

Data Cube Architecture

Data

Data CubesUsers• Working with CEOS Space Agencies to develop requirements andplans for sustained provision of Analysis Ready Data (ARD)• Landsat, Sentinels, MODIS, SRTM, ALOS, SPOT, and more ....

• Open source software, developed and sustained by CEOS
• Support for diverse datasets and grid projections• Deployment via local computers, regional hubs (e.g. SERVIR), orcomputing cloud (e.g. Amazon)• Connections to common GIS tools (ArcGIS, QGIS)• Advanced Programming Interfaces (APIs) for users• Prototypes in Colombia and Kenya• Developing and testing user interfaces for custom mosaic creation,water detection and other concepts• Investigating many capacity building options

Custom Mosaic Tool

Filter Selections
Country: Kenya or Colombia
Data Product: L7 or L8
Product Type: 11 options
Non-clear pixels: RED flag
Season: Continuous or Multi
Dates: Month-Year
Bounding Box or Lat-Long

Data Cube Water Detection Tool
11-Year Analysis Results
Blue = frequent water
Red/Yellow = infrequent
flood events
Flood risk can be easily
inferred from the 23-step
Australian WOFS algorithm.
30-meter Landsat resolution
allows detailed assessmentsthat are far better than
MODIS (250-m).

Country Demonstrations• Prototype Data Cube projects are under development in Colombia andKenya• The Colombia project supported by CSIRO and IDEAM (Institute ofHydrology, Meteorology and Environmental Studies).
• A Colombia mini-cube (4 Landsat path-row regions) was delivered in Oct
2015. 20 time-series stacked tiles (1-deg square) with data back to 2000.
• The Colombia team is growing capacity ... Colombia has alreadydemonstrated use of the ingestor software to expand the data in theircube and they have modified the user interface to add median mosaics,
PCA change detection and NDVI-based forest/non-forest maps. They areinterested in support more applications such as water resource
management.
• The Kenya project is supported by the Australian Government and theClinton Foundation (CCI and SLEEK).
• Two operating versions of the Kenya Data Cube exist: Amazon Cloud anda local SEO computer. We are also considering options for the SERVIRAfrica hub. The Kenya Data Cube is 11.5 TB of Landsat data (7500+ scenes
back to 2000) from 42 path-row regions put into 68 time-series stackedtiles (1-deg square).
• The Kenya team is currently utilizing scene-based methods to develop
historic forest maps. They will move to a Data Cube approach in 2017.

Technical Details
 Data Cube team is moving to a version-2 implementation by
mid-2016. NetCDF4-CF1.6 cube format vs. GeoTIFF. Many
updates to the APIs and ingestor code.
 Additional development of user interface modules, along withtesting, is planned by the end of 2016.
 Deployment to any country should be possible by early 2017.
All code will be available from the Data Cube GitHub site.
 Ingestor code (creates data cube): Python + libraries
 Installation time: 8-12 hours to configure and install on AWS
Ingestion rate: 100-250 Landsat scenes per day on AWS.
 Storage Size: ~ 25% of size of input data (compression) Database type (data cube query): PostgreSQL Spatial grid options: UTM, Sinusoidal, Albers Equal Area
 APIs (3): Storage Query and Access, Analytics, Execution User interface modules: Custom Mosaics, Water Detection, andmore under consideration (TBD)