Package: gis-metadata-parser
gis-metadata-parserXML parsers for GIS metadata that are designed to read in, validate, update and output a core set of properties that have been mapped between the most common standards, currently:FGDCISO-19139 (and ISO-19115)ArcGIS (tested with ArcGIS format 1.0).This library is compatible with Python versions 2.7 and 3.4 through 3.6.InstallationInstall withpip install gis-metadata-parser.UsageParsers can be instantiated from files, XML strings or URLs. They can be converted from one standard to another as well.fromgis_metadata.arcgis_metadata_parserimportArcGISParserfromgis_metadata.fgdc_metadata_parserimportFgdcParserfromgis_metadata.iso_metadata_parserimportIsoParserfromgis_metadata.metadata_parserimportget_metadata_parser# From file objectswithopen(r'/path/to/metadata.xml')asmetadata:fgdc_from_file=FgdcParser(metadata)withopen(r'/path/to/metadata.xml')asmetadata:iso_from_file=IsoParser(metadata)# Detect standard based on root element, metadatafgdc_from_string=get_metadata_parser("""<?xml version='1.0' encoding='UTF-8'?><metadata><idinfo></idinfo></metadata>""")# Detect ArcGIS standard based on root element and its nodesiso_from_string=get_metadata_parser("""<?xml version='1.0' encoding='UTF-8'?><metadata><dataIdInfo/></dataIdInfo><distInfo/></distInfo><dqInfo/></dqInfo></metadata>""")# Detect ISO standard based on root element, MD_Metadata or MI_Metadataiso_from_string=get_metadata_parser("""<?xml version='1.0' encoding='UTF-8'?><MD_Metadata><identificationInfo></identificationInfo></MD_Metadata>""")# Convert from one standard to anotherfgdc_converted=iso_from_file.convert_to(FgdcParser)iso_converted=fgdc_from_file.convert_to(IsoParser)arcgis_converted=iso_converted.convert_to(ArcGISParser)# Output supported properties as key value pairs (dict)fgdc_key_vals=fgdc_from_file.convert_to(dict)iso_key_vals=iso_from_file.convert_to(dict)Finally, the properties of the parser can be updated, validated, applied and output:withopen(r'/path/to/metadata.xml')asmetadata:fgdc_from_file=FgdcParser(metadata)# Example simple propertiesfgdc_from_file.titlefgdc_from_file.abstractfgdc_from_file.place_keywordsfgdc_from_file.thematic_keywords# :see: gis_metadata.utils.SUPPORTED_PROPS for list of all supported properties# Complex propertiesfgdc_from_file.attributesfgdc_from_file.bounding_boxfgdc_from_file.contactsfgdc_from_file.datesfgdc_from_file.digital_formsfgdc_from_file.larger_worksfgdc_from_file.process_stepsfgdc_from_file.raster_info# :see: gis_metadata.utils.COMPLEX_DEFINITIONS for structure of all complex properties# Update propertiesfgdc_from_file.title='New Title'fgdc_from_file.dates={'type':'single''values':'1/1/2016'}# Apply updatesfgdc_from_file.validate()# Ensure updated properties are validfgdc_from_file.serialize()# Output updated XML as a stringfgdc_from_file.write()# Output updated XML to existing filefgdc_from_file.write(out_file_or_path='/path/to/updated.xml')# Output updated XML to new fileExtending and CustomizingTipsThere are a few unwritten (until now) rules about the way the metadata parsers are wired to work:Properties are generally defined by XPATH in eachparser._data_mapSimple parser properties accept only values ofstringandlist's ofstring'sXPATH's configured in the data map support references to element attributes:'path/to/element/@attr'Complex parser properties are defined by custom parser/updater functions instead of by XPATHComplex parser properties accept values of typedictcontaining simple properties, or a list of saiddict'sXPATH keys in the data map with leading underscores are parsed, but not validated or written outXPATH keys in the data map that "shadow" other properties but with a leading underscore serve as secondary valuesSecondary values are used in the absence of a primary value if primary location (element or attribute) is missingAdditional underscores indicate further locations to check for missing values, i.e.title,_title,__titleSome examples of existing secondary properties are as follows:# In the ArcGIS parser for distribution contact phone:ARCGIS_TAG_FORMATS=frozendict({...'dist_phone':'distInfo/distributor/distorCont/rpCntInfo/cntPhone/voiceNum','_dist_phone':'distInfo/distributor/distorCont/rpCntInfo/voiceNum',# If not in cntPhone...})# In the FGDC parser for sub-properties in the contacts definition:FGDC_DEFINITIONS=dict({k:dict(v)fork,viniteritems(COMPLEX_DEFINITIONS)})FGDC_DEFINITIONS[CONTACTS].update({'_name':'{_name}','_organization':'{_organization}'})...classFgdcParser(MetadataParser):...def_init_data_map(self):...ct_format=FGDC_TAG_FORMATS[CONTACTS]fgdc_data_structures[CONTACTS]=format_xpaths(...name=ct_format.format(ct_path='cntperp/cntper'),_name=ct_format.format(ct_path='cntorgp/cntper'),# If not in cntperporganization=ct_format.format(ct_path='cntperp/cntorg'),_organization=ct_format.format(ct_path='cntorgp/cntorg'),# If not in cntperp)# Also see the ISO parser for secondary and tertiary sub-properties in the attributes definition:ISO_DEFINITIONS=dict({k:dict(v)fork,viniteritems(COMPLEX_DEFINITIONS)})ISO_DEFINITIONS[ATTRIBUTES].update({'_definition_source':'{_definition_src}','__definition_source':'{__definition_src}','___definition_source':'{___definition_src}'})ExamplesAny of the supported parsers can be extended to include more of a standard's supported data. In this example we'll add two new properties to theIsoParser:metadata_language: a simple string field describing the language of the metadata file itself (not the dataset)metadata_contacts: a complex structure with contact info leveraging and enhancing the existing contact structureThis example will cover:Adding a new simple propertyConfiguring a secondary location for a property valueReferencing an element attribute in an XPATHAdding a new complex propertyCustomizing the complex property to include a new sub-propertyAlso, this example is specifically covered by unit tests.fromgis_metadata.iso_metadata_parserimportIsoParserfromgis_metadata.utilsimportCOMPLEX_DEFINITIONS,CONTACTS,format_xpaths,ParserPropertyclassCustomIsoParser(IsoParser):def_init_data_map(self):super(CustomIsoParser,self)._init_data_map()# 1. Basic property: text or list (with secondary location referencing `codeListValue` attribute)lang_prop='metadata_language'self._data_map[lang_prop]='language/CharacterString'# Parse from here if presentself._data_map['_'+lang_prop]='language/LanguageCode/@codeListValue'# Otherwise, try from here# 2. Complex structure (reuse of contacts structure plus phone)# 2.1 Define some basic variablesct_prop='metadata_contacts'ct_xpath='contact/CI_ResponsibleParty/{ct_path}'ct_defintion=COMPLEX_DEFINITIONS[CONTACTS]ct_defintion['phone']='{phone}'# 2.2 Reuse CONTACT structure to specify locations per prop (adapted from parent to add `phone`)self._data_structures[ct_prop]=format_xpaths(ct_defintion,name=ct_xpath.format(ct_path='individualName/CharacterString'),organization=ct_xpath.format(ct_path='organisationName/CharacterString'),position=ct_xpath.format(ct_path='positionName/CharacterString'),phone=ct_xpath.format(ct_path='contactInfo/CI_Contact/phone/CI_Telephone/voice/CharacterString'),email=ct_xpath.format(ct_path='contactInfo/CI_Contact/address/CI_Address/electronicMailAddress/CharacterString'))# 2.3 Set the contact root to insert new elements at "contact" level given the defined path:#   'contact/CI_ResponsibleParty/...'# By default we would get multiple "CI_ResponsibleParty" elements under a single "contact"# This way we get multiple "contact" elements, each with its own single "CI_ResponsibleParty"self._data_map['_{prop}_root'.format(prop=ct_prop)]='contact'# 2.4 Leverage the default methods for parsing complex properties (or write your own parser/updater)self._data_map[ct_prop]=ParserProperty(self._parse_complex_list,self._update_complex_list)# 3. And finally, let the parent validation logic know about the two new custom propertiesself._metadata_props.add(lang_prop)self._metadata_props.add(ct_prop)withopen(r'/path/to/metadata.xml')asmetadata:iso_from_file=CustomIsoParser(metadata)iso_from_file.metadata_languageiso_from_file.metadata_contacts