From XML to Excel for Data Analysis | by Costas Andreou | Towards Data Science

Depending on your role and company, now and again you may be required to work with XML. In this article, we will explore how you can leverage Python to assist you in extracting and shaping data in such a way as to enable you to carry out your analysis.
XML stands for eXtensible Markup Language, and it’s a language designed primarily to store and transfer data. The language is designed in such a way so that it’s both machine and human-readable. Critics of the language will often propose JSON as an alternative, suggesting faster processing times.
The XML language is made up of tags, elements and attributes. The best way to understand each of those terms and how we can use them is through the usage of examples.
When it comes to processing XML in Python, the most popular library is suitably named XML. You can consult the relevant documentation here. Were you to visit the documentation page, you’d find that there are several sub-modules available for handling XML, but for our purposes, we will solely be using xml.etree.ElementTree.
First up, let us brush on our XML knowledge. Let us look at the following:
<catalog> <product description="Cardigan Sweater">  <catalog_item gender="Men's">   <item_number>QWZ5671</item_number>   <price>39.95</price>  </catalog_item> </product><catalog>
In the above example, we notice the following:
The root element is catalog, and there can only exist one
All tags are case sensitive, and all elements need to have a closing tag (such as price)
Product is a child element, catalog is a sub-child element, item_number and price a sub-sub-child.
For our examples, I will be using this sample XML file; with the only difference that I repeated the product sub-child element twice.
The first thing we need to do is import the library and read in the XML file into our application:
import xml.etree.ElementTree as ET#Example XML files:#   sample2 from:  https://www.service-architecture.com/articles/object-oriented-databases/xml_file_for_complex_data.htmltree = ET.parse(r'C:\xml\sample2.xml')root = tree.getroot()tag = root.tagatt = root.attribprint(root,'\n',tag,'\n',att)
which returns the following:
<Element 'catalog' at 0x0093FC60> catalog {}
Hence, we can see how we can identify the root node, which has no attribute. Let us then go a level lower on the XML tree, using:
for child in root:    print(child.tag, child.attrib)
which returns:
product {'description': 'Cardigan Sweater', 'product_image': 'cardigan.jpg'}product {'description': 'Cardigan Sweater', 'product_image': 'cardigan.jpg'}
That’s great. We can now identify tags as well as their attributes. What if we wanted to extract the values? Let’s go two more levels deeper where we have such examples (for instance, at the <price> element):
for child in root:    #print(child.tag, child.attrib)    for subchild in child:        #print(subchild.tag, subchild.attrib, subchild.text)        for subsubchild in subchild:            print(subsubchild.tag, subsubchild.attrib, subsubchild.text)
Giving us:
item_number {} QWZ5671price {} 39.95size {'description': 'Medium'}size {'description': 'Large'}
Empowered with the above knowledge, you should now be in a position to easily navigate the XML tree and extract any information that you want. This doesn’t help you much; however, when you’re trying to carry out analysis on the data within the XML tree.
In some instances, you simply want the data in a tabular format so you can quickly draw some conclusions. Maybe you are attempting to see the unique values across a specific tag, or perhaps you are merely trying to understand the data better. Maybe you want to showcase the data in a way your stakeholders are more familiar with — and what is more familiar to a business user than excel.
As such, what we want to use next is the iter() method of the xml.etree.ElementTree python library. This method will allow us to iterate over every XML element:
for elem in root.iter():    print(elem.tag, elem.attrib, elem.text)
which returns, as you’d expect a list such as the following:
catalog {}product {'description': 'Cardigan Sweater', 'product_image': 'cardigan.jpg'}catalog_item {'gender': "Men's"}item_number {} QWZ5671price {} 39.95size {'description': 'Medium'}
Assuming we would like to create a line for each product (i.e. the first level child), we can use the following script:
import xml.etree.ElementTree as ETtree = ET.parse(r'pathToXML')root = tree.getroot()tag = root.tagatt = root.attrib#Flatten XML to CSVfor child in root:    mainlevel = child.tag    xmltocsv = ''    for elem in root.iter():        if elem.tag == root.tag:            continue        if elem.tag == mainlevel:            xmltocsv = xmltocsv + '\n'        xmltocsv = xmltocsv + str(elem.tag).rstrip() + str(elem.attrib).rstrip() + ';' + str(elem.text).rstrip() + ';'print(xmltocsv)
which file, we can then open with Excel:
Worth noting at this point, that for the data to be aligned (between each line), it is assumed that all XML attributes will be present for each child node. If that is not the case, then there might be some cleaning up of the data required.
I hope you find this useful.
If you liked this article, you might also like: