Perl Testing in 2023
Tags: perl, tdd, test2suite
Published: 2023-01-24 17:58:46

---
title: Perl Testing in 2023
published: true
date: 2023-01-24 17:38:36 UTC
tags: Perl,perl,TDD,Test2Suite
canonical_url: https://toby.ink/blog/2023/01/24/perl-testing-in-2023/
cover_image: https://toby.ink/wp-content/uploads/2023/01/pexels-thisisengineering-3862132-1024x683.jpg
---

With my open source work, I’ve historically taken an approach which relies more on integration testing than unit testing, but with some of my newer projects, I’ve tried adopting principles from $paidwork and applying them to my free software.

This is a quick run-down of how I’m structuring my test suite in newer projects. It’s likely that many of my existing projects will never adopt this structure, but some may.

## Out with the old, in with the new

First step is ditching [Test::More](https://metacpan.org/pod/Test%3A%3AMore), [Test::Fatal](https://metacpan.org/pod/Test%3A%3AFatal), [Test::Warnings](https://metacpan.org/pod/Test%3A%3AWarnings), and other pre-Test2 testing libraries.

[Test2::V0](https://metacpan.org/pod/Test2%3A%3AV0) provides a good base to work with, so we add that to our project’s requirements list straight away.

We’ll round that out with:

- [Test2::Require::AuthorTesting](https://metacpan.org/pod/Test2%3A%3ARequire%3A%3AAuthorTesting) to skip certain tests when run on the end user’s machine, and require the `AUTHOR_TESTING` environment variable to run. This is useful for tests which are very slow or require a highly specific environment to run in.
- [Test2::Require::Module](https://metacpan.org/pod/Test2%3A%3ARequire%3A%3AModule) to skip certain tests when optional modules are unavailable.
- [Test2::Tools::Spec](https://metacpan.org/pod/Test2%3A%3ATools%3A%3ASpec) to better structure our unit tests.
- [Test2::Plugin::BailOnFail](https://metacpan.org/pod/Test2%3A%3APlugin%3A%3ABailOnFail) for when your tests simply cannot carry on. Use sparingly.

There are other Test2 modules that can be nice to have, but that’s a good starter set. The ones I list above are distributed alongside [Test2::V0](https://metacpan.org/pod/Test2%3A%3AV0), so you get them “for free”. It is worth explicitly listing them in your project’s dependencies though, in case they are split into separate distributions in the future.

## Test directory structure

Perl distributions typically keep their tests in a directory called _t_ found inside the project root. We won’t change that. We will however create two subdirectories within it, _t/unit_ and _t/integration_.

You may have other categories of tests which need their own subdirectories too, but these two will be sufficient for most projects.

If support modules are needed for testing, they can live in _t/lib_. If any data files are needed, put them in _t/share_.

Any especially important tests can included in the _t_ directory itself for maximum visibility. I recommend giving them numerically-prefixed filenames for better control over the order they run in.

I will typically include two such tests:

- _t/00start.t_ which performs no real testing, but prints relevant information about the system it’s being run on, such as the version numbers of dependencies, the values of important environment variables, etc.
- _t/01basic.t_ which loads all of your modules, or at least the important ones, then passes without any real testing being done. The purpose of this is to quickly check for syntax errors so extreme that they prevent your code from even compiling. This is a good place for [Test2::Plugin::BailOnFail](https://metacpan.org/pod/Test2%3A%3APlugin%3A%3ABailOnFail).

## Unit tests

For each module _lib/Foo/Bar.pm_ there should be a corresponding unit test script _t/unit/Foo/Bar.t_.

The prelude of this file will look something like this:

```perl
  use Test2::V0 -target => 'Foo::Bar';
  use Test2::Tools::Spec;
  use Data::Dumper;
```

The footer will look exactly like this:

```perl
  done_testing;
```

And in between will be tests for each unit of code. A “unit” would typically be a function, a method, or an important package variable. Tests for a unit should avoid exercising much code outside their unit, and should _especially_ avoid exercising code outside the target module.

A fairly extensive unit test for a method:

```perl
  describe "method `foo_bar`" => sub {
    
    my ( $foo, $bar, $expected_foo_bar );
    
    case 'both defined' => sub {
      $foo = 'Hello';
      $bar = 'world';
      $expected_foo_bar = 'Hello world';
    };
    
    case 'both defined, but foo is empty string' => sub {
      $foo = '';
      $bar = 'world';
      $expected_foo_bar = ' world';
    };
    
    case 'both defined, but bar is empty string' => sub {
      $foo = 'Hello';
      $bar = '';
      $expected_foo_bar = 'Hello ';
    };
    
    case 'both defined, but both are empty string' => sub {
      $foo = '';
      $bar = '';
      $expected_foo_bar = ' ';
    };
    
    case 'foo is undefined' => sub {
      $foo = undef;
      $bar = 'world';
      $expected_foo_bar = 'world';
    };
    
    case 'bar is undefined' => sub {
      $foo = 'Hello';
      $bar = undef;
      $expected_foo_bar = 'Hello';
    };
    
    case 'both are undefined' => sub {
      $foo = undef;
      $bar = undef;
      $expected_foo_bar = '';
    };
    
    tests 'it works' => sub {
      my ( $object, $got_foo_bar, $got_exception, $got_warnings );
      
      $got_exception = dies {
        $got_warnings = warns {
          $object = $CLASS->new( foo => $foo, bar => $bar );
          $got_foo_bar = $object->foo_bar;
        };
      };
      is( $got_exception, undef, 'no exception thrown' );
      is( $got_warnings, 0, 'no warnings generated' );
      is( $got_foo_bar, $expected_foo_bar, 'expected string returned' );
      is(
        $object,
        object {
          call foo => $foo;
          call bar => $bar;
        },
        "method call didn't alter the values of the attributes",
      ) or diag Dumper( $object );
    };
  };
```

The module itself as a whole can also be considered a “unit” so that very basic module-wide concerns can be tested there.

A fairly simple module-wide unit test:

```perl
  describe "class `$CLASS`" => sub {
    
    tests 'it inherits from Moo::Object' => sub {
      isa_ok( $CLASS, 'Moo::Object' );
    };
    
    tests 'it can be instantiated' => sub {
      can_ok( $CLASS, 'new' );
    };
  };
```

## Integration tests

Integration tests can take a far more freeform approach. These tests should ensure that the system as a whole, or subsystems within it (which may involve multiple modules), work when used together.

A good place to start is to look at the _SYNOPSIS_ sections of your documentation and test that they work as advertised.

Integration tests should use [Test2::V0](https://metacpan.org/pod/Test2%3A%3AV0) but will often be small enough not to benefit much from [Test2::Tools::Spec](https://metacpan.org/pod/Test2%3A%3ATools%3A%3ASpec).

## Running tests locally

The Test2 tool for running your test suite is [yath](https://metacpan.org/pod/App%3A%3AYath), but old stalwart [prove](https://metacpan.org/pod/App%3A%3AProve) also works well. In your project roor directory, you should be able to run `yath test` at the command-line to run your entire test suite. `prove -lr` should also work.

If you need more verbose output to see exactly which tests are passing, then append `-v` to each command.

## Continuous integration with GitHub Actions

I will normally use GitHub Actions to automatically run my test suite on each push, on every major version of Perl I support. One of the test runs will load [Devel::Cover](https://metacpan.org/pod/Devel%3A%3ACover) and use it to upload test coverage data to [Codecov](https://codecov.io) and [Coveralls](https://coveralls.io).

This gives me (almost) instant feedback on whether recent commits have broken things or reduced test coverage.

100% coverage on Coveralls is totally achieveable. 100% coverage on Codecov often takes a lot more work as it measures branch coverage instead of statement coverage. It’s certainly a good goal though.

Exactly how to set up continuous integration will depend a lot on your built tools, so I won’t go into specifics here.

## Summary

I feel this setup provides a pretty good basis for test-driven Perl development in 2023.