Perling and Curling
Tags: perl, programming
Published: 2021-05-07 04:36:06

Most of us probably know [curl](https://curl.se/) as a quick and easy way to send HTTP requests from the command line.

That tool, though, is just an interface to the curl project’s real gold: the libcurl API. Using this API, applications in all sorts of languages have easy access to the awesome power that libcurl provides. This article will discuss how to use that power in Perl.

# A Quick Example

```
use Net::Curl::Easier;

my $easy = Net::Curl::Easier->new(
    url => 'http://perl.org',
    followlocation => 1,
)->perform();

print $easy->head(), $easy->body();
```
Let’s talk about what just happened.

[Net::Curl::Easier](https://metacpan.org/pod/Net::Curl::Easier) is a thin wrapper around [Net::Curl](https://metacpan.org/pod/Net::Curl)’s “easy” interface—[“easy” is what libcurl calls it!](https://curl.se/libcurl/c/libcurl-easy.html)—that smooths over some rough edges in Net::Curl.

(Full disclosure: I am Net::Curl::Easier’s maintainer.)

Once we create our “Easier” object, having given it the proper URL and told it to follow HTTP redirects (`followlocation` refers to [HTTP’s `Location` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Location)), we run `perform()` on the Easier object.

After that, we print the HTTP response headers and body, and we’re done!

# Why not just use [HTTP::Tiny](https://metacpan.org/pod/HTTP::Tiny)?

Indeed. Well, error reporting, for one. Consider:
```
Net::Curl::Easier->new(
    url => 'http://blahblah',
)->perform();
```
If you run this you’ll probably just see `Couldn't resolve host name` printed to standard error. But if you dig deeper you’ll see something nifty:
```
use Net::Curl::Easier;
use Data::Dumper;

eval {
    Net::Curl::Easier->new(
        url => 'http://blahblah',
    )->perform();
};
print Dumper $@;
```
It turns out that that error isn’t just a string; it’s an exception _object_.

In large systems I often want to handle certain failure types differently from others. HTTP::Tiny’s errors are just strings, so type-specific failure handling with HTTP::Tiny entails _parsing strings_, which is brittle. What if someone decides to reword some error message for clarity, thus breaking my string parser?

With Net::Curl I can look for specific numeric error codes, documentation for which [the curl project itself maintains](https://curl.se/libcurl/c/libcurl-errors.html). This is much more robust.

# Don’t care. What else you got?

OK. How about this:
```
my $easy = Net::Curl::Easier->new(
    username => 'hal',
    userpwd => 'itsasecret',
    url => 'imap://mail.example.com/INBOX/;UID=123',
)->perform();
```
I just queried … an email inbox?!?

Curl doesn’t just speak HTTP; it speaks many other protocols including IMAP, LDAP, SCP, and MQTT. To see the full list of protocols that your curl supports, run `curl --version`.

# Concurrency

Curl can also run concurrent queries. To do that I recommend using [Net::Curl::Promiser](https://metacpan.org/pod/Net::Curl::Promiser). (Full disclosure: I also maintain this module.)

Example, assuming use of [Mojolicious](http://metacpan.org/pod/Mojolicious):
```
use Net::Curl::Easier;
use Net::Curl::Promiser::Mojo;
use Mojo::Promise;

my $easy1 = Net::Curl::Easier->new(
    url => 'http://perl.org',
    followlocation => 1,
);

my $easy2 = Net::Curl::Easier->new(
    username => 'hal',
    userpwd => 'itsasecret',
    url => 'imap://mail.example.com/INBOX/;UID=123',
);

my $easy3 = Net::Curl::Easier->new(
    username => 'hal',
    userpwd => 'itsasecret',
    url => 'scp://tty.example.com/path/to/file',
);

my $promiser = Net::Curl::Promiser::Mojo->new();

Mojo::Promise->all_settled(
    $promiser->add_handle($easy1)->then( sub {
        print $easy1->head(), $easy1->body();
    } ),
    $promiser->add_handle($easy2)->then( sub {
        # ... whatever you want with the IMAP result
    } ),
    $promiser->add_handle($easy3)->then( sub {
        # ... whatever you want with the SCP result
    } ),
)->wait();
```
We just grabbed a web page, queried a mailbox, and downloaded a file via SCP, all in parallel!

Note, too, that this method interfaces seamlessly with other promises. So if you have existing [Mojo::UserAgent](http://metacpan.org/pod/Mojo::UserAgent)-based code, you can add requests for other protocols alongside it.

Net::Curl::Promiser also works natively with
[AnyEvent](http://metacpan.org/pod/AnyEvent) and
[IO::Async](http://metacpan.org/pod/IO::Async), should those be of greater interest to you. It also provides a convenience layer for custom [select](https://perldoc.perl.org/perlfunc#select-RBITS%2CWBITS%2CEBITS%2CTIMEOUT)-based event loops, in case that’s how you roll.

# Other Modules

Some alternatives to modules presented above:

* [AnyEvent::YACurl](https://metacpan.org/pod/AnyEvent::YACurl): A newer library than Net::Curl that simplifies the interface a bit. It assumes use of [AnyEvent](https://metacpan.org/pod/AnyEvent), though, so if you’re not using AE then this may not be for you.

* [WWW::Curl](https://metacpan.org/pod/WWW::Curl): The library of which Net::Curl is a fork. It can do much of what Net::Curl does but lacks access to libcurl’s [MULTI_SOCKET interface](https://curl.se/libcurl/c/libcurl-multi.html), which is faster and more flexible than curl’s internal [select](https://linux.die.net/man/2/select)-based manager for concurrent requests.

* [Net::Curl::Simple](https://metacpan.org/pod/Net::Curl::Simple): A wrapper by Net::Curl’s original author. It provides some of the same conveniences as Net::Curl::Promiser and Net::Curl::Easier but uses callbacks rather than promises.

# Closing Thoughts

Curl exposes an awesome breadth of functionality, of which the above examples have just scratched the surface. Check it out!