Package: pg-to-evalscript
openEO process graph to evalscript converterThis repository contains a library for convertingopenEO process graphstoSentinel Hub evalscripts.The motivation behind this library is to reduce the data transfer between the SH backend and the openEO backend and to move part of the processing directly to backend where the data is stored.APIconvert_from_process_graphconvert_from_process_graph(process_graph,user_defined_processes={},n_output_bands=1,sample_type="FLOAT32",units=None,bands_dimension_name="bands",temporal_dimension_name="t",bands_metadata=[])Parametersprocess_graph:dictOpenEO process graph JSON as Python dict object.user_defined_processes:dictDictionary ofuser-defined processesthat can be used in the process graph.The format should be:<user-defined-process-id> : <user-defined-process-graph>.n_output_bands:int, optional. Default:1Number of output bands in the evalscript. This can be set if the value is known beforehand. Seedocs.sample_type:str, optional. Default:FLOAT32DesiredsampleTypeof the output raster. Seepossible values.units:str, optional. Default:NoneUnits used by all the bands in the evalscript. IfNone,unitsevalscript parameter isn't set and default units for each band are used. Seedocs.bands_dimension_name:str, optional. Default:bandsName of the default dimension of typebandsof the datacube, as set inload_collectionand referred to in the openEO process graph.temporal_dimension_name:str, optional. Default:tName of the default dimension of typetemporalof the datacube, as set inload_collectionand referred to in the openEO process graph.bands_metadata:list, optional. Default:[]List of metadata information for all bands of a certain collection.encode_result:bool, optional. Default:TrueShould the result of the evalscript be encoded with the dimensions of the data or returned as is.Outputevalscripts:list of dictsReturns a list of dicts containing theEvalscriptobjects. Every element consists of:invalid_node_id: Id of the first invalid node after a supported subgraph. The output of the associated evalscript should be the input of the node. If it isNone, the entire graph is valid.evalscript: instance of andEvalscriptobject that generates an evalscript for a valid subgraph fromload_collectionto the node with idinvalid_node_id.EvalscriptConstructor parametersinput_bands:listList of bands to be imported. Seedocs.nodes:listList ofNodeobjects that constitute the valid process (sub)graph.initial_data_name:strThe id of the initialload_collectionnode that loads the data.n_output_bands:int, optional. Default:1Number of output bands in the evalscript. This can be set if the value is known beforehand. Seedocs.sample_type:str, optional. Default:FLOAT32DesiredsampleTypeof the output raster. Seepossible values.units:str, optional. Default:NoneUnits used by all the bands in the evalscript. IfNone,unitsevalscript parameter isn't set and default units for each band are used. Seedocs.mosaicking:str, optional. Default:ORBITWorks with multi-temporal data by default. Seepossible values.bands_dimension_name:str, optional. Default:bandsName of the default dimension of typebandsof the datacube, as set inload_collectionand referred to in the openEO process graph.temporal_dimension_name:str, optional. Default:tName of the default dimension of typetemporalof the datacube, as set inload_collectionand referred to in the openEO process graph.datacube_definition_directory:str, optional. Default:javascript_datacubeRelative path to the directory with the javascript implemetations of the processes.output_dimensions:list of dicts or None, optional. Default:NoneInformation about the dimensions in the output datacube. This can be set if the value is known beforehand. Each element contains:name: name of the dimension,size: size (length) of the dimension,original_temporal, optional: boolean, should beTrueif this is the temporal dimension generated in the initialload_collectionnode.encode_result:bool, optional. Default:TrueShould the result of the evalscript be encoded with the dimensions of the data or returned as is.bands_metadata:list, optional. Default:[]List of metadata information for all bands of a certain collection.Methodswrite():Returns the evalscript as a string.determine_output_dimensions():Calculates the greatest possible dimensions of the output datacube, returning a list of dicts. Each element contains:name: name of the dimension,size: size (length) of the dimension,original_temporal, optional: boolean,Trueif this is the temporal dimension generated in the initialload_collectionnode.set_output_dimensions(output_dimensions):Setter for output dimensions.output_dimensionsis a list of dicts. Each element contains:name: name of the dimension,size: size (length) of the dimension,original_temporal, optional: boolean, should beTrueif this is the temporal dimension generated in the initialload_collectionnode.set_input_bands(input_bands):Setter for input bands.input_bandsis an array of strings (band names) orNone. Output dimensions are recalculated.get_decoding_function():Returns adecode_datafunction. The data returned by the evalscript is encoded to contain the information about the datacube dimensions and has to be decoded to obtain the actual data in a ndarray format.decode_datahas the following parameters:data: the result of processing of the associated evalscript, it should be a three-dimensional array.decode_datareturns a multidimensional Python list.list_supported_processesReturns a list of process ids of supportedopenEO processes.WorkflowConstruct the openEO process graphLoad a file withjson.loador generate an openEO process graph usingopenEO Python client.Run the conversionsubgraph_evalscripts=convert_from_process_graph(process_graph)print(subgraph_evalscripts)>>>[{'evalscript':<pg_to_evalscript.evalscript.Evalscriptobjectat0x000001ABA779CA00>,'invalid_node_id':None}]In this example, the entire openEO process graph could be converted to an evalscript, so we only have one entry.Fetch the dataevalscript=subgraph_evalscripts[0]['evalscript'].write()print(evalscript)>>>"//VERSION=3 function setup(){ ..."The evalscript string can now be used to process data on Sentinel Hub.Sentinel Hub Python clientmakes it easy to do so.Decode the fetched data# Get the decoding function fo  r this evalsscriptdecoding_function=evalscript.get_decoding_function()# Pass the fetched data through the decoding function.# The function expects a python list. If you're using Sentinel Hub Python client, the result might be a numpy array, so it has to be converted.decoded_data=decoding_function(fetched_data.tolist())print(decoded_data)>>>[[[1,2,3],[4,5,6],...]]Running notebookspipenv install --dev --preStart the notebookscd notebooks
pipenv run jupyter notebookRunning testsUsing Dockerdocker build -t tests .
docker run testsDirectlyTests require a NodeJS environment. Use version =< 8.2.1 to match Sentinel Hub behaviour.pipenv install
pipenv shell
cd tests
pytestLintingpipenv run black -l 120 .Benchmarkpipenv shell
cd tests
python benchmark.pyDevelopingInstall the package in editable mode so the changes take effect immediately.pipenv install -e .