Package: lidaco
LidacoLidaco (WindLidarDataConverter) is a library and executable that enables a modular writing of data converters.Following the configurations that are specified by the user on a config.yml(s), a Reader module is selected to import the data from input files.
Similarly, a Writer is also selected to write the output file(s).Lidaco works on datasets that can be described using the unidataCommon Data Model. It can be used to process single files or entire folders.Available Readers* AQ500
* Galion
* WLS70
* Windcubev1
* Windcubev2
* Windscanner
* ZephIR300
* TritonAvailable Writers* MetadataCard
* NcML
* NetCDF4Getting startedInstallpipinstalllidacoRun in the command line (shell)lidaco--config-file=samples/Windscanner/config.yamlUse as a libraryfromlidaco.core.BuilderimportBuilderbuilder=Builder(config_file='path/to/config.yaml')builder.build()Converting dataWriting conversion filesEach conversion requires a configuration file written inYAML. This file contains four (optional) main groups of configurations:ParametersThis section allows you to specify the parameters for the converter itself, the selected reader and or writer.
The converter parameters are:parameters:input:path:./path/to/input/folder/format:Windscanner# The name should matchoutput:path:./path/to/output/folder# optional, defaults to ./output/format:NetCDF4# Optional, and specifies the number of input files to be concatenated per output file.# If not specified defaults to 1 (1 output per input file). The last output will contain <= output_block_size input files.# If the parameter is specified with None (the same as without any value), all input files will be concatenated into a single output file.output_block_size:3Some of these parameters can be overridden using the command arguments, runlidaco --helpto know them.When specifying relative paths, the configuration file location wherein these are specified, or the current directory when specifying them as command arguments, is the one used to resolve its absolute value.Additional parameters might be necessary depending on the specific reader/writer in use. Read the respective module documentation available, or try to take a look at its source code.AttributesThis section specify the global attributes that will be added to the dataset. You can add all attributes you desire.attributes:# e.g.,lidar_technology:'pulsed'lidar_scanning_type:'verticalprofiling'data_processing_history:'datatakenfrom.scnfilesgeneratedby...'VariablesThis section is similar toAttributesbut instead of reading being used to specify Attributes, it is used to specify variables.variables:# e.g.,pitch:data_type:'f4'units:'degrees'long_name:'lidar_pitch_angle'comment:''accuracy:''accuracy_info:'Noinformationonpitchaccuracyavailable.'value:0ImportsLidaco configuration system is built to motivate the configurations reusability, so each config file can import others. With this mechanism you can to split your configurations into devices, scenarios, campaigns and so on.
In case of multiple definitions of an attribute or variable, the definitions in the file prevail over the others that are imported.imports:# read in order-./general/NEWA_Kassel_general_dataset.yaml-./instruments/Windcubev2_general_instrument_description.yaml-./processing/NEWA_Kassel_data_processing_history.yamlFor more examples on how to setup your config files, take a look at the availableSamples.ContributingIf you would like to add, or see being added, some change to the converter you can:Open an issueso that it is discussed by everyone.And/or alsoSubmit a pull requestwith the desired changes. (About pull requests)To submit a new Reader/WriterCreate a python class at lidaco.readers or lidaco.writers, if you are writing a reader or writer respectively.The file and class should have the same name, that will be used in the config files.It should extend core.reader or core.writer.Take a look at the existingreadersandwriters.