ooici/coi-services — pull request #297 by lukecampbell (2012-10-10 14:33:22+00:00)
Title: Transforms

Data Processes and Stream Definitions with Bindings
Abstract

Each of our transforms have an output binding key word. For some transforms, there can be more than one binding, for others it's an intuitive symbol that resolves to an unambiguous intention for "what is to be sent on this stream." Inside the transforms, the actual data process portion can use multiple bindings to send different granule types on different streams. For example take this hypothetical CTD multiplexer.

conductivity_granule = ConductivityTransform.execute(incoming_granule)
temperature_granule = TemperatureTransfrom.execute(incoming_granule)
depth_granule = DepthTransform.execute(incoming_granule)

self.conductivity.publish(conductivity_granule)
self.temperature.publish(temperature_granule)
self.depth.publish(depth_granule)
Changes

We will start including a list of output bindings as class variables for our transforms.

The interface to DataProcessManagementService::assign_stream_definition_to_data_process_definition changed to include an option for the bindings.

The data process definition now has a field for output bindings, it is a dictionary which is set up with assign_stream_definition_to_data_process_definition. Clients will specify a stream definition to be bound to a specific binding key for a transform. For example the salinity output for a transform may have a certain type of parameter dictionary/stream definition so clients can specify individual stream definitions for each output of a transform.

Here is an example

ctd_pdict_id = self.dataset_management.read_parameter_dictionary_by_name('ctd_parsed_param_dict', id_only=True)
sal_stream_def_id = self.pubsubclient.create_stream_definition(name='Salinity', parameter_dictionary_id=ctd_pdict_id)
self.dataprocessclient.assign_stream_definition_to_data_process_definition(sal_stream_def_id, ctd_L2_salinity_dprocdef_id, binding='salinity' )
Affects

Clients to SA must be mindful of the bindings a transform has.

We now support multi-output transforms, so anything that makes assumptions about the number of outputs for a transform need to accommodate this change.

Workflow Management just takes the first output data product of a transform and uses that for workflows, we may want to look at this again.
Data Process Management clients no longer need to specify a parameter dictionary, they can simply provide the stream definition.