Package: asf-metadata
asf_metadataThis repo contains all the Python 3.8 scripts related to manage ASF metadata. These are currently mostly dealing with ISO metadata.Table of ContentsISO templateExcel spreadsheetISO metdata functionsISO metadata toolsISO templateISO metadata templates can be best maintained in a more user-friendly environment. I have settled for an Excel spreadsheet to achieve that.The template mechanism has three different flows. All use an Excel spreadsheet to define the structure of the ISO metadata.ISO metadata template generationThe ISO metadata's structure is stored in an Excel spreadsheet and contains identifiers for each metadata parameter that is variable. The structure and variables form the template that we can store in two formats: XML and JSON.Format conversionIn order to convert the metadata between various formats, we need to know the structure of it. That is saved within an Excel spreadsheet.This functionality is something I anticipate using for the NISAR mission, for which we only get metadata in ISO XML format. Using the structure from the spreadsheet allows to painlessly deal with the namespaces within the XML files.ISO metadata generationFor the generation of ISO metadata, we need the structure from the Excel spreadsheet. The template parameters need to be exchanged by their actual values. This will be achieved by extracting the necessary information based on the product files, replacing the functionality that existed in the old MapReady framework.Excel spreadsheetThe Excel spreadsheet has a number of worksheets.ISO metadata structureThis worksheet provides the relevant parts of the ISO metadata, most importantly its structure and the namespaces used within that structure.NamespacesOne of the biggest nuisances about ISO XML metadata is the namespaces.This worksheet links each parameter to its respective namespace.This worksheet defines the URLs for the individual namespaces needed in the XML declarations.ISO metadata functionsI developed a number of functions that perform the various tasks outline above.Extracting information from templatesThe template information in the Excel spreadsheet is extracted (using theisoTemplate2listsfunction) into a number of lists: the flattened template structure, the individual metadata parameters, and their respective values.Conversion of metadata formatsIn order to convert an XML file to JSON format, we need to extract the individual metadata parameters out of the ISO metadata XML file using theget_metadata_valuesfunction.Replacing placeholders with actual valuesThe templates stored in the Excel spreadsheets contain ISO parameter placeholders. Theproduct_dictionary2valuesfunction replaces these placeholders, saved in a product dictionary, with the actual metadata values.Generating ISO metadata from product filesISO metadata can be generated by extracting the necessary information based on the product files (using the generate_product_dictionary function). Depending on the product type, specific conversion functions are called:RTC products:gamma_rtc_metadatafunctionBuilding metadata structuresThe ISO metadata is saved in two different formats: XML and JSON. The XML is using a tree structure that is built with theiso_xml_structurefunction, while the JSON structure is formed by a complex nested dictionary using theiso_dictionary_structurefunction.Writing metadata to fileThere are two separate functions to save the metadata to a file format.meta_xml_fileis the function that generates an ISO XML metadata file, whilemeta_json_filesaves the metadata in JSON format.Cleaning up metadataThere are two separate functions to clean up the metadata structures.clean_xml_structure(for XML files) andclean_json_structure(for JSON files) is the function that removes metadata for product files (digital elevation model, incidence angle map, scattering area map) that are only optionally included.ISO metadata toolsI developed a number of tools that perform the various tasks outline above.Generate ISO template file from Excel spreadsheetusage: generate_iso_template.py [-h] [-dem DEM] excelFile isoBaseGenerate ISO template file from Excel spreadsheetpositional arguments:
excelFile   name of the Excel template spreadsheet
isoBase     basename of the ISO metadata template fileoptional arguments:
-h, --help  show this help message and exit
-dem DEM    name of DEM template spreadsheetGenerate ISO metadata file from Excel spreadsheetusage: generate_iso_metadata.py [-h] [-product PRODUCT] [-data DATA] [-dem DEM] meta_file log_file iso_baseGenerate ISO metadata file from Excel spreadsheetpositional arguments:
meta_file         file path of product metadata such as a "manifest.safe" or leader file
log_file          name of RTC processing log file
iso_base          basename of the ISO XML metadata fileoptional arguments:
-h, --help        show this help message and exit
-product PRODUCT  name of the product type (default: gamma_rtc)
choices: ["gamma_rtc"]
-data DATA        name of the data source (default: sentinel)
choices: ["sentinel"]
-dem DEM          DEM type used for terrain correction
choices: ["copernicus"]At the moment, I have only added one functiongamma_rtc_log2metathat analyzes GAMMA RTC log files for metadata. The corresponding product type is 'gamma_rtc'.AuthorRudi Gens