Package: geodataflow
GeodataFlowGeospatial processing framework for geographical & Earth Observation (EO) data in Python.GeodataFlowis a Geoprocessing framework for fetching, translating and manipulating Geospatial data (Raster,Vector,EO/STAC collections) by using aPipelineor sequence of operations on input data. It is very much like theGDALlibrary which handles raster and vector data.The project is split up into several namespace packages or components:geodataflow.coreThe main subpackage ofGeodataFlowwhich implements basic building blocks (Pipeline engine & Modules) and commonly used functionalities.geodataflow.apiWebAPI component usingFastAPIwhich provides access to GeodataFlow backend via API REST calls.workbench/uiGeodataFlowWorkbenchis a staticjavascriptapplication for users easily draw and run their own Workflows in the Web Browser.NOTE: There is no any installer for GeodataFlowWorkbenchyet, but you can test it loading thedocker-compose.yml. Please, read related section below.Backends:spatialInstalls thegeodataflow.spatialbackend implementation for GeodataFlow usingGDAL/OGR.dataframesInstalls thegeodataflow.dataframesbackend implementation for GeodataFlow usingGeopandas.pySpark,Geospatial SQL,... ?Videos demostratingGeodataFlow:What is GeodataFlowTranforming FeaturesReading items from a STAC CatalogDownloading a NDVI raster from a STAC CatalogFiltering Features applying Spatial RelationsPlotting a NDVI Time Series Graph of a polygonWorkflow examplesAssuming you are usinggeodataflow.spatial(GDAL/OGR) as active backend implementation,GeodataFlowcan run workflows as the following:Converting a Shapefile to GeoPackage:# ==============================================================# Pipeline sample to convert a Shapefile to GeoPackage.# =============================================================={"pipeline":[{"type":"FeatureReader","connectionString":"input.shp"},# Extract the Centroid of input geometries.{"type":"GeometryCentroid"},# Transform CRS of geometries.{"type":"GeometryTransform","sourceCrs":4326,"targetCrs":32630},# Save features to Geopackage.{"type":"FeatureWriter","connectionString":"output.gpkg"}]}Fetching metadata of a S2L2A Product (STAC):# ==============================================================# Pipeline sample to fetch metadata of a S2L2A Product (STAC).# =============================================================={"pipeline":[{"type":"FeatureReader",# Define the input AOI in an embedded GeoJson."connectionString":{"type":"FeatureCollection","crs":{"type":"name","properties":{"name":"EPSG:4326"}},"features":[{"type":"Feature","properties":{"id":0,"name":"My AOI for testing"},"geometry":{"type":"Polygon","coordinates":[[[-1.746826,42.773227],[-1.746826,42.860866],[-1.558685,42.860866],[-1.558685,42.773227],[-1.746826,42.773227]]]}}]}},# Transform CRS of geometries.{"type":"GeometryTransform","sourceCrs":4326,"targetCrs":32630},# Fetch metadata of EO Products that match one SpatioTemporial criteria.{"type":"EOProductCatalog","driver":"STAC","provider":"https://earth-search.aws.element84.com/v0/search","product":"sentinel-s2-l2a-cogs","startDate":"2021-09-25","endDate":"2021-10-05","closestToDate":"2021-09-30","filter":"","preserveInputCrs":true},# Save features to Geopackage.{"type":"FeatureWriter","connectionString":"output.gpkg"}]}InstallationBecauseGeodataFlowis composed by several namespace packages, some of them are optional (e.g. Backend implementations). You will need to install the ones you want by adding them as an extra to the command-line that runs the installer.In order to read and write Cloud Optimized Geotiffs (COG), GDAL version 3.1 or greater is required. If your system GDAL is older than version 3.1, consider using Docker or Conda to get a modern GDAL.Using pypiTo install the latest stable version frompypi, write this in the command-line:>pipinstallgeodataflow[api,dataframes,eodag,gee]Thegeodataflowpackage installsgeodataflow.coreandgeodataflow.spatialones by default. You can use namespace package installers as well  (e.g.api), they have the same effect than the generic one.Optional extras for Backends:eodagEODAG -Earth Observation Data Access Gatewayis a Python package for searching and downloading remotely sensed images while offering an unified API for data access regardless of the data provider.geeGEE -Google Earth Engine APIis a geospatial processing service. WithEarth Engine, you can perform geospatial processing at scale, powered by Google Cloud Platform.GEErequires authentication, please, read available documentationhere.To view all available CLI tool commands and options:>geodataflow--helpListing all available modules:>geodataflow--modulesRun a workflow in the command-line interface:>geodataflow--pipeline_file"/geodataflow/spatial/tests/data/test_eo_stac_catalog.json"Using docker-composedocker-compose.ymlbuilds images and starts  GeodataFlow API and Workbench components to easily run Workflows withGeodataFlow.PACKAGE_WITH_GEODATAFLOW_PIPELINE_CONTEXTin the yml file indicates the backend implementation to load. The default value isgeodataflow.spatial. If you prefer to use another backend, please, change it before starting.Write in the command-line from the root folder of the project:>docker-composeupThen, type in your favorite Web Browser:http://localhost:9630/docsto check the interface of the REST WebAPI service.http://localhost:9640/workbench.htmlto access to the Workbench UI designer and where you can design and run Workflows!To remove all resources:>docker-composedown--rmiall-v--remove-orphansTestingEach package provides a collection of tests, run tests ontestsfolders to validate them.ContributeHave you spotted a typo in our documentation? Have you observed a bug while runningGeodataFlow? Do you have a suggestion for a new feature?Don't hesitate and open an issue or submit a pull request, contributions are most welcome!LicenseGeodataFlowis licensed under Apache License v2.0.
SeeLICENSEfile for details.CreditsGeodataFlowis built on top of amazingly useful open source projects. SeeNOTICEfile for details about those projects
and their licenses.Thank you to all the authors of these projects!AuthorsGeodataFlowhas been created byAlvaro Huartehttps://www.linkedin.com/in/alvarohuarte.