Package: tilebench
tilebenchInspect HEAD/LIST/GET requests within RasterioSource Code:https://github.com/developmentseed/tilebenchInspect HEAD/GET requests withing Rasterio.Note: In GDAL 3.2, logging capabilities for /vsicurl, /vsis3 and the like was added (ref:https://github.com/OSGeo/gdal/pull/2742).InstallYou can installtilebenchusing pip$python-mpipinstall-Upip
$python-mpipinstall-Utilebenchor install from source:gitclonehttps://github.com/developmentseed/tilebench.gitcdtilebench

python-mpipinstall-Upip
python-mpipinstall-e.APIfromtilebenchimportprofileimportrasterio[@profile](https://github.com/profile)()definfo(src_path:str):withrasterio.open(src_path)assrc_dst:returnsrc_dst.metameta=info("https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif")>2023-10-18T23:00:11.184745+0200|TILEBENCH|{"HEAD":{"count":1},"GET":{"count":1,"bytes":32768,"ranges":["0-32767"]},"Timing":0.7379939556121826}fromtilebenchimportprofilefromrio_tiler.ioimportReader[@profile](https://github.com/profile)()def_read_tile(src_path:str,x:int,y:int,z:int,tilesize:int=256):withReader(src_path)ascog:returncog.tile(x,y,z,tilesize=tilesize)img=_read_tile("https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif",9114,13216,15,)>2023-10-18T23:01:00.572263+0200|TILEBENCH|{"HEAD":{"count":1},"GET":{"count":2,"bytes":409600,"ranges":["0-32767","32768-409599"]},"Timing":1.0749869346618652}Command Line Interface (CLI)$ tilebench --help
Usage: tilebench [OPTIONS] COMMAND [ARGS]...

  Command line interface for the tilebench Python package.

Options:
  --help  Show this message and exit.

Commands:
  get-zooms  Get Mercator Zoom levels.
  profile    Profile COGReader Mercator Tile read.
  random     Get random tile.
  viz        WEB UI to visualize VSI statistics for a web mercator tile requestExamples$ tilebench get-zooms https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif | jq
{
  "minzoom": 14,
  "maxzoom": 19
}

$ tilebench random https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --zoom 15
15-9114-13215

$ tilebench profile https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --tile 15-9114-13215 --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR | jq
{
  "HEAD": {
    "count": 1
  },
  "GET": {
    "count": 2,
    "bytes": 409600,
    "ranges": [
      "0-32767",
      "32768-409599"
    ]
  },
  "Timing": 0.9715230464935303
}

$ tilebench profile https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --tile 15-9114-13215 --config GDAL_DISABLE_READDIR_ON_OPEN=FALSE | jq
{
  "HEAD": {
    "count": 8
  },
  "GET": {
    "count": 3,
    "bytes": 409600,
    "ranges": [
      "0-32767",
      "32768-409599"
    ]
  },
  "Timing": 2.1837549209594727
}Starlette MiddlewareWarning: This is highly experimental and should not be used in production (https://github.com/developmentseed/tilebench/issues/6)In addition of thevizCLI we added a starlette middleware to easily integrate VSI statistics in your web services.fromfastapiimportFastAPIfromtilebench.middlewareimportVSIStatsMiddlewareapp=FastAPI()app.add_middleware(VSIStatsMiddleware)The middleware will add avsi-statsentry in the responseheadersin form of:vsi-stats: list;count=1, head;count=1, get;count=2;size=196608, ranges; values=0-65535|65536-196607Some paths may be excluded from being handeld by the middleware by theexclude_pathsargument:app.add_middleware(VSIStatsMiddleware,exclude_paths=["/foo","/bar"])GDAL config optionsCPL_TIMESTAMP: Add timings on GDAL LogsGDAL_DISABLE_READDIR_ON_OPEN: Allow or Disable listing of files in the directory (e.g external overview)GDAL_INGESTED_BYTES_AT_OPEN: Control how many bytes GDAL will ingest when opening a dataset (useful when a file has a big header)CPL_VSIL_CURL_ALLOWED_EXTENSIONS: Limit valid external filesGDAL_CACHEMAX: Cache sizeGDAL_HTTP_MERGE_CONSECUTIVE_RANGESVSI_CACHEVSI_CACHE_SIZESee the full list athttps://gdal.org/user/configoptions.htmlInternal tiles Vs Mercator grid$ tilebench viz https://noaa-eri-pds.s3.amazonaws.com/2022_Hurricane_Ian/20221002a_RGB/20221002aC0795145w325100n.tif --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIRBlue lines represent the mercator grid for a specific zoom level and the red lines represent the internal tiles boundsWe can then click on a mercator tile and see how much requests GDAL/RASTERIO does.DockerReady to use docker image can be found on Github registry.https://github.com/developmentseed/tilebench/pkgs/container/tilebenchdockerrun\--volume"$PWD":/data\--platformlinux/amd64\--rm-it-p8080:8080ghcr.io/developmentseed/tilebench:latest\tilebenchviz--host0.0.0.0https://noaa-eri-pds.s3.us-east-1.amazonaws.com/2020_Nashville_Tornado/20200307a_RGB/20200307aC0865700w360900n.tifContribution & DevelopmentSeeCONTRIBUTING.mdLicenseSeeLICENSEAuthorsSeecontributorsfor a listing of individual contributors.ChangesSeeCHANGES.md.Changelog0.12.0 (2024-01-24)allowtmsoptions in CLI (profile,randomandget-zooms) to select TileMatrixSet0.11.0 (2023-10-18)update requirementsrio-tiler>=6.0,<7.0fastapi>=0.100.0rasterio>=1.3.8removewurlitzerdependencyonly userasteriologsremoveLISTinformationbreaking change0.10.0 (2023-06-02)updaterio-tilerrequirementfix log parsing whenCPL_TIMESTAMP=ONis set0.9.1 (2023-03-24)handle dateline crossing dataset and remove pydantic serialization0.9.0 (2023-03-14)update pre-commit and fix issue with starlette>=0.26re-writeNoCacheMiddlewareas pure ASGI middlewarerenameanalyse_logstoparse_logsadd python 3.11 support0.8.2 (2022-11-21)update hatch config0.8.1 (2022-10-31)fix issue with min/max zoom when there is no overviewscalculate windows from block_shapes0.8.0 (2022-10-25)update rio-tiler/rasterio dependenciesremove python 3.7 supportadd python 3.10 supportadd image endpoint to show the data footprintswitch from mapbox to maplibre0.7.0 (2022-06-14)addcProfilestats0.6.1 (2022-04-19)Remove usage ofVSIStatsMiddlewareintilebench viz0.6.0 (2022-04-19)switch to pyproject.toml0.5.1 (2022-03-04)make sure we don't cache previous request when usingtilebench profilewithout--tileoption0.5.0 (2022-02-28)update rio-tiler requirementaddreaderoption0.4.1 (2022-02-14)update Fastapi requirementuse WarpedVRT to get dataset bounds in epsg:43260.4.0 (2021-12-13)update rio-tiler's version requirementadd more information about the raster in the Viz web page (author@drnextgis,https://github.com/developmentseed/tilebench/pull/14)fix bug for latest GDAL/rasterio versionadd default STAMEN basemap invizand remove mapbox token/style options.update fastapi requirement0.3.0 (2021-03-05)addexclude_pathsoptions inVSIStatsMiddlewareto exclude some endpoints (author@drnextgis,https://github.com/developmentseed/tilebench/pull/10)renamedressourcestoresources0.2.1 (2021-02-19)fix typo in UI0.2.0 (2021-01-28)add warp-kernels in output inprofileCLIadd rasterio/curl stdout in outputadd dataread time in Viz0.1.1 (2021-01-27)update requirements0.1.0 (2021-01-04)add web UI for VSI stats visualizationadd starlette middleware0.0.2 (2020-12-15)Update for rio-tiler==2.0.0rc30.1.0 (2020-07-13)Initial release