Perl and XML in 2021: A few lessons learned
Tags: programming, perl, work, xml
Published: 2021-03-28 01:35:21

---
title: Perl and XML in 2021: A few lessons learned
published: true
date: 2021-03-28 01:26:40 UTC
tags: Programming,Perl,work,XML
canonical_url: https://phoenixtrap.com/2021/03/27/perl-and-xml-in-2021-a-few-lessons-learned/?utm_source=rss&utm_medium=rss&utm_campaign=perl-and-xml-in-2021-a-few-lessons-learned
---

It’s been years since I’ve had to hack on anything [XML](https://en.wikipedia.org/wiki/XML)-related, but a recent project at work has me once again jumping into the waters of generating, parsing, and modifying this 90s-era document format. Most developers these days likely only know of it as part of the curiously-named [XMLHTTPRequest](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest) object in web browsers used to retrieve data in [JSON](https://en.wikipedia.org/wiki/JSON) format from servers, and as the “X” in [AJAX](https://en.wikipedia.org/wiki/Ajax_%28programming%29). But here we are in 2021, and there are still plenty of APIs and documents using XML to get their work done.

In my particular case, the task is to update the API calls for a new version of [Virtuozzo Automator](https://www.virtuozzo.com/support/all-products/virtuozzo-automator.html). Its API is a bit unusual in that it doesn’t use [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol), but rather relies on opening a [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security)-encrypted socket to the server and exchanging documents delimited with a [null character](https://en.wikipedia.org/wiki/Null_character). The previous version of our code is in 1990s-sysadmin-style Perl, with manual [`bless`](https://perldoc.pl/functions/bless)ing of objects and parsing the XML using [regular expressions](https://perldoc.pl/perlre). I’ve decided to update it to use the [Moo](https://metacpan.org/pod/Moo) object system and a proper XML parser. But which parser and module to use?

## Selecting a parser

There are several [generic XML modules](https://perlmaven.com/xml) for parsing and generating XML on [CPAN](https://www.cpan.org/), each with its own advantages and disadvantages. I’d like to say that I did a comprehensive survey of each of them, but this project is pressed for time _(aren’t they all?)_ and I didn’t want to create too many extra dependencies in our Perl stack. Luckily, [XML::LibXML](https://metacpan.org/pod/XML::LibXML) is already available, I’ve had some previous experience with it, and it’s a good choice for performant standards-based XML parsing (using either [DOM](https://metacpan.org/pod/distribution/XML-LibXML/lib/XML/LibXML/DOM.pod) or [SAX](https://metacpan.org/pod/distribution/XML-LibXML/lib/XML/LibXML/SAX.pod)) and generation.

Given more time and leeway in adding dependencies, I might use something else. If the Virtuozzo API had an [XML Schema](https://en.wikipedia.org/wiki/XML_schema) or used [SOAP](https://en.wikipedia.org/wiki/SOAP), I would consider [XML::Compile](https://metacpan.org/pod/XML::Compile) as I’ve had some success with that in other projects. But even that uses XML::LibXML under the hood, so I’d still be using that. Your mileage may vary.

## Generating XML

Depending on the size and complexity of the XML documents to generate, you might choose to build them up node by node using [XML::LibXML::Node](https://metacpan.org/pod/distribution/XML-LibXML/lib/XML/LibXML/Node.pod) and [XML::LibXML::Element](https://metacpan.org/pod/distribution/XML-LibXML/lib/XML/LibXML/Element.pod) objects. Most of the messages I’m sending to Virtuozzo Automator are short and have easily-interpolated values, so I’m using [here-document](https://perldoc.pl/perlop#%3C%3CEOF) islands of XML inside my Perl code. This also has the advantage of being easily validated against the examples in the documentation.

Where the interpolated values in the messages are a little complicated, I’m using [this idiom](https://metacpan.org/pod/distribution/perlsecret/lib/perlsecret.pod#Baby-cart) inside the here-docs:

```perl
@{[...]}
```

This allows me to put an arbitrary expression in the … part, which is then put into an anonymous array reference, which is then immediately dereferenced into its string result. It’s a cheap and cheerful way to do minimal templating inside Perl strings without loading a full templating library; I’ve also had success using this technique when generating [SQL](https://en.wikipedia.org/wiki/SQL) for database queries.

## Parser as an object attribute

Rather than instantiate a new [XML::LibXML](https://metacpan.org/pod/XML::LibXML) in every method that needs to parse a document, I created a private attribute:

```perl
package Local::API::Virtozzo::Agent {
    use Moo;
    use XML::LibXML;
    use Types::Standard qw(InstanceOf);
    ...
    has _parser => (
        is => 'ro',
        isa => InstanceOf['XML::LibXML'],
        default => sub { XML::LibXML->new() },
    );
    sub foo {
        my $self = shift;
        my $send_doc = $self->_parser
          ->parse_string(<<"END_XML");
            <foo/>
END_XML
        ...
    }
...
}
```

## Boilerplate

XML documents can be verbose, with elements that rarely change in every document. In the Virtuozzo API’s case, every document has a `<packet>` element containing a `version` attribute and an `id` attribute to match requests to responses. I wrote a simple function to wrap my documents in this element that pulled the version from a constant and always increased the `id` by one every time it’s called:

```perl
sub _wrap_packet {
    state $send_id = 1;
    return qq(<packet version="$PACKET_VERSION" id=")
      . $send_id++ . '">' . shift . '</packet>';
}
```

If I need to add more attributes to the `<packet>` element (for instance, namespaces for attributes in enclosed elements, I can always use [XML::LibXML::Element::setAttribute](https://metacpan.org/pod/distribution/XML-LibXML/lib/XML/LibXML/Element.pod#setAttribute) after parsing the document string.

## Parsing responses with [XPath](https://en.wikipedia.org/wiki/XPath)

Rather than using brittle regular expressions to extract data from the response, I use the shared parser object from above and then the full power of [XPath](https://en.wikipedia.org/wiki/XPath):

```perl
use English;
...
sub get_sampleID {
    my ($self, $sample_name) = @_;
    ...
    # used to separate documents
    local $INPUT_RECORD_SEPARATOR = "\0";
    # $self->_sock is the IO::Socket::SSL connection
    my $get_doc = $self->_parser( parse_string(
      $self->_sock->getline(),
    ) );
    my $sample_id = $get_doc->findvalue(
        qq(//ns3:id[following-sibling::ns3:name="$sample_name"]),
    );
    return $sample_id;
}
```

This way, even if the order of elements change or more elements are introduced, the XPath patterns will continue to find the right data.

## Conclusion… so far

I’m only about halfway through updating these API calls, and I’ve left out some non-XML-related details such as setting up the TLS socket connection. Hopefully this article has given you a taste of what’s involved in XML processing these days. Please leave me a comment if you have any suggestions or questions.