Package: rio-tiler-pds
Rio-Tiler-PDS: A rio-tiler plugin for Public DatasetsA rio-tiler plugin to read from publicly-available datasets.ImportantThis is the new module for rio-tiler missions specific (ref:https://github.com/cogeotiff/rio-tiler/issues/195)Documentation:https://cogeotiff.github.io/rio-tiler-pds/Source Code:https://github.com/cogeotiff/rio-tiler-pdsInstallationYou can install rio-tiler-pds using pip$pipinstall-Upip
$pipinstallrio-tiler-pdsor install from source:$pipinstall-Upip
$pipinstallgit+https://github.com/cogeotiff/rio-tiler-pds.gitDatasetsDataLevel/ProductFormatOwnerRegionBucket TypeSentinel 2L1CJPEG2000Sinergise / AWSeu-central-1Requester-paysSentinel 2L2AJPEG2000Sinergise / AWSeu-central-1Requester-paysSentinel 2L2ACOGDigital Earth Africa / AWSus-west-2PublicSentinel 1L1C GRD (IW, EW, S1-6)COG (Internal GCPS)Sinergise / AWSeu-central-1Requester-paysLandsat Collection 2L1,L2COGUSGS / AWSus-west-2Requester-paysCBERS 4/4AL2/L4COGAMS Kepler / AWSus-east-1Requester-paysMODIS (modis-pds)MCD43A4, MOD09GQ, MYD09GQ, MOD09GA, MYD09GAGTiff (External Overviews)-us-west-2PublicMODIS (astraea-opendata)MCD43A4, MOD11A1, MOD13A1, MYD11A1 MYD13A1COGAstraea / AWSus-west-2Requester-paysCopernicus Digital Elevation ModelGLO-30, GLO-90COGSinergise / AWSeu-central-1PublicAdding more dataset:If you know of another publicly-available dataset that can easily be described
with a "scene id", please feel free toopen an
issue.WarningsRequester-pays BucketsOn AWS,sentinel2,sentinel1,cbersandmodis(in astraea-opendata) datasets are stored inrequester
paysbuckets. This means that the cost of GET and LIST requests and egress fees for
downloading files outside the AWS region will be charged to theaccessing
users, not the organization hosting the bucket. Forrio-tilerandrio-tiler-pdsto work with such buckets, you'll need to setAWS_REQUEST_PAYER="requester"in your shell environment.Partial reading on Cloud hosted datasetWhen reading data,rio-tiler-pdsperformspartialreads when possible. Hence
performance will be best on data stored asCloud Optimized GeoTIFF
(COG). It's important to note thatSentinel-2 scenes hosted
on AWS are not in Cloud Optimized format but in JPEG2000. Partial reads from
JPEG2000 files are inefficient, and GDAL (the library underlyingrio-tiler-pdsandrasterio) will need to makemany GET requestsand transfer a lot of
data. This will be both slow and expensive, since AWS's JPEG2000 collection of
Sentinel 2 data is stored in a requester pays bucket.Ref:Do you really want people using your datablog post.OverviewReadersEach dataset has its own submodule (e.g sentinel2:rio_tiler_pds.sentinel.aws)fromrio_tiler_pds.landsat.awsimportLandsatC2Readerfromrio_tiler_pds.sentinel.awsimportS1L1CReaderfromrio_tiler_pds.sentinel.awsimport(S2JP2Reader,# JPEG2000S2COGReader,# COG)fromrio_tiler_pds.cbers.awsimportCBERSReaderfromrio_tiler_pds.modis.awsimportMODISPDSReader,MODISASTRAEAReaderfromrio_tiler_pds.copernicus.awsimportDem30Reader,Dem90ReaderAll Readers are subclass ofrio_tiler.io.BaseReaderand inherit its properties/methods.Propertiesbounds: Scene bounding boxcrs: CRS of the bounding boxgeographic_bounds: bounding box in geographic projection (e.g WGS84)minzoom: WebMercator MinZoom (e.g 7 for Landsat 8)maxzoom: WebMercator MaxZoom (e.g 12 for Landsat 8)Methodsinfo: Returns band's simple info (e.g nodata, band_descriptions, ....)statistics: Returns band's statistics (percentile, histogram, ...)tile: Read web mercator map tile from bandspart: Extract part of bandspreview: Returns a low resolution preview from bandspoint: Returns band's pixel value for a given lon,latfeature: Extract part of bandsOtherbands(property): List of available bands for each datasetScene IDAll readers take scene id as main input. Thescene idis used internaly by the reader to derive the full path of the data.e.g: Landsat on AWSBecause the Landsat AWS PDS follows a regular schema to store the data (s3://{bucket}/c1/L8/{path}/{row}/{scene}/{scene}_{band}.TIF"), we can easily reconstruct the full band's path by parsing the scene id.fromrio_tiler_pds.landsat.awsimportLandsatC2Readerfromrio_tiler_pds.landsat.utilsimportsceneid_parsersceneid_parser("LC08_L2SP_001062_20201031_20201106_02_T2")>{'sensor':'C','satellite':'08','processingCorrectionLevel':'L2SP','path':'001','row':'062','acquisitionYear':'2020','acquisitionMonth':'10','acquisitionDay':'31','processingYear':'2020','processingMonth':'11','processingDay':'06','collectionNumber':'02','collectionCategory':'T2','scene':'LC08_L2SP_001062_20201031_20201106_02_T2','date':'2020-10-31','_processingLevelNum':'2','category':'standard','sensor_name':'oli-tirs','_sensor_s3_prefix':'oli-tirs','bands':('QA_PIXEL','QA_RADSAT','SR_B1','SR_B2','SR_B3','SR_B4','SR_B5','SR_B6','SR_B7','SR_QA_AEROSOL','ST_ATRAN','ST_B10','ST_CDIST','ST_DRAD','ST_EMIS','ST_EMSD','ST_QA','ST_TRAD','ST_URAD')}withLandsatC2Reader("LC08_L2SP_001062_20201031_20201106_02_T2")aslandsat:print(landsat._get_band_url("SR_B2"))>s3://usgs-landsat/collection02/level-2/standard/oli-tirs/2020/001/062/LC08_L2SP_001062_20201031_20201106_02_T2/LC08_L2SP_001062_20201031_20201106_02_T2_SR_B2.TIFEach dataset has a specific scene id format:!!! note "Scene ID formats"- Landsat
    - link: [rio_tiler_pds.landsat.utils.sceneid_parser](https://github.com/cogeotiff/rio-tiler-pds/blob/e4421d3cf7c23b7b3552b8bb16ee5913a5483caf/rio_tiler_pds/landsat/utils.py#L35-L56)
    - regex: `^L[COTEM]0[0-9]_L\d{1}[A-Z]{2}_\d{6}_\d{8}_\d{8}_\d{2}_(T1|T2|RT)$`
    - example: `LC08_L1TP_016037_20170813_20170814_01_RT`

- Sentinel 1 L1C
    - link: [rio_tiler_pds.sentinel.utils.s1_sceneid_parser](https://github.com/cogeotiff/rio-tiler-pds/blob/e4421d3cf7c23b7b3552b8bb16ee5913a5483caf/rio_tiler_pds/sentinel/utils.py#L98-L121)
    - regex: `^S1[AB]_(IW|EW)_[A-Z]{3}[FHM]_[0-9][SA][A-Z]{2}_[0-9]{8}T[0-9]{6}_[0-9]{8}T[0-9]{6}_[0-9A-Z]{6}_[0-9A-Z]{6}_[0-9A-Z]{4}$`
    - example: `S1A_IW_GRDH_1SDV_20180716T004042_20180716T004107_022812_02792A_FD5B`

- Sentinel 2 JPEG2000 and Sentinel 2 COG
    - link: [rio_tiler_pds.sentinel.utils.s2_sceneid_parser](https://github.com/cogeotiff/rio-tiler-pds/blob/e4421d3cf7c23b7b3552b8bb16ee5913a5483caf/rio_tiler_pds/sentinel/utils.py#L25-L60)
    - regex: `^S2[AB]_[0-9]{2}[A-Z]{3}_[0-9]{8}_[0-9]_L[0-2][A-C]$` or `^S2[AB]_L[0-2][A-C]_[0-9]{8}_[0-9]{2}[A-Z]{3}_[0-9]$`
    - example: `S2A_29RKH_20200219_0_L2A`, `S2A_L1C_20170729_19UDP_0`, `S2A_L2A_20170729_19UDP_0`

- CBERS
    - link: [rio_tiler_pds.cbers.utils.sceneid_parser](https://github.com/cogeotiff/rio-tiler-pds/blob/e4421d3cf7c23b7b3552b8bb16ee5913a5483caf/rio_tiler_pds/cbers/utils.py#L28-L43)
    - regex: `^CBERS_(4|4A)_\w+_[0-9]{8}_[0-9]{3}_[0-9]{3}_L\w+$`
    - example: `CBERS_4_MUX_20171121_057_094_L2`, `CBERS_4_AWFI_20170420_146_129_L2`, `CBERS_4_PAN10M_20170427_161_109_L4`, `CBERS_4_PAN5M_20170425_153_114_L4`, `CBERS_4A_WPM_20200730_209_139_L4`

- MODIS (PDS and Astraea)
    - link: [rio_tiler_pds.modis.utils.sceneid_parser](https://github.com/cogeotiff/rio-tiler-pds/blob/c533d38330f46738c46cb9927dbe91b299dc643d/rio_tiler_pds/modis/utils.py#L29-L42)
    - regex: `^M[COY]D[0-9]{2}[A-Z0-9]{2}\.A[0-9]{4}[0-9]{3}\.h[0-9]{2}v[0-9]{2}\.[0-9]{3}\.[0-9]{13}$`
    - example: `MCD43A4.A2017006.h21v11.006.2017018074804`Band Per Asset/Filerio-tiler-pdsReaders assume that bands (e.g eo:band in STAC) are stored in separate files.$awss3lss3://usgs-landsat/collection02/level-2/standard/oli-tirs/2020/001/062/LC08_L2SP_001062_20201031_20201106_02_T2/--request-payer
LC08_L2SP_001062_20201031_20201106_02_T2_ANG.txt
LC08_L2SP_001062_20201031_20201106_02_T2_MTL.json
LC08_L2SP_001062_20201031_20201106_02_T2_MTL.txt
LC08_L2SP_001062_20201031_20201106_02_T2_MTL.xml
LC08_L2SP_001062_20201031_20201106_02_T2_QA_PIXEL.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_QA_RADSAT.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B1.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B2.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B3.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B4.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B5.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B6.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_B7.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_QA_AEROSOL.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_SR_stac.json
LC08_L2SP_001062_20201031_20201106_02_T2_ST_ATRAN.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_B10.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_CDIST.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_DRAD.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_EMIS.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_EMSD.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_QA.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_TRAD.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_URAD.TIF
LC08_L2SP_001062_20201031_20201106_02_T2_ST_stac.json
LC08_L2SP_001062_20201031_20201106_02_T2_thumb_large.jpeg
LC08_L2SP_001062_20201031_20201106_02_T2_thumb_small.jpegWhen reading data or metadata, readers will merge them.e.gwithS2COGReader("S2A_L2A_20170729_19UDP_0")assentinel:img=sentinel.tile(78,89,8,bands=("B01","B02"))assertimg.data.shape==(2,256,256)stats=sentinel.statistics(bands=("B01","B02"))print(stats)>>{'B01':BandStatistics(min=2.0,max=17132.0,mean=2183.7570706659685,count=651247.0,sum=1422165241.0,std=3474.123975478363,median=370.0,majority=238.0,minority=2.0,unique=15112.0,histogram=[[476342.0,35760.0,27525.0,24852.0,24379.0,23792.0,20891.0,13602.0,3891.0,213.0],[2.0,1715.0,3428.0,5141.0,6854.0,8567.0,10280.0,11993.0,13706.0,15419.0,17132.0]],valid_percent=62.11,masked_pixels=397329.0,valid_pixels=651247.0,percentile_2=179.0,percentile_98=12465.0),'B02':BandStatistics(min=1.0,max=15749.0,mean=1941.2052554560712,count=651247.0,sum=1264204099.0,std=3130.545395156859,median=329.0,majority=206.0,minority=11946.0,unique=13904.0,histogram=[[479174.0,34919.0,27649.0,25126.0,24913.0,24119.0,20223.0,12097.0,2872.0,155.0],[1.0,1575.8,3150.6,4725.4,6300.2,7875.0,9449.8,11024.6,12599.4,14174.199999999999,15749.0]],valid_percent=62.11,masked_pixels=397329.0,valid_pixels=651247.0,percentile_2=134.0,percentile_98=11227.079999999958)}print(stats["B01"].min)>>2.0Mosaic Reader: Copernicus DEMThe Copernicus DEM GLO-30 and GLO-90 readers are notper scenebutmosaicreaders. This is possible because the dataset is a global dataset with file names having thegeo-locationof the COG, meaning we can easily contruct a filepath from a coordinate.fromrio_tiler_pds.copernicus.awsimportDem30ReaderwithDem30Reader()asdem:print(dem.assets_for_point(-57.2,-11.2))>>['s3://copernicus-dem-30m/Copernicus_DSM_COG_10_S12_00_W058_00_DEM/Copernicus_DSM_COG_10_S12_00_W058_00_DEM.tif']ChangesSeeCHANGES.md.Contribution & DevelopmentSeeCONTRIBUTING.mdLicenseSeeLICENSE.txtAuthorsThe rio-tiler project was begun at Mapbox and has been transferred in January 2019.SeeAUTHORS.txtfor a listing of individual contributors.