Distar - A CPAN Distribution Author Tool
Tags: perl, cpan
Published: 2021-03-31 01:42:35

Basic CPAN Tooling
==================

When releasing a distribution on [CPAN][cpan], authors need to include an installer script. Classically, this would be a `Makefile.PL` using [ExtUtils::MakeMaker][eumm] to generate a `Makefile`, which would then be used with `make`. Later, [Module::Build][mb] was created, using a `Build.PL` script which served a similar purpose. The installer script would include information about the distribution, such as the name and prerequisites. This script, as well as the information inside it, would be used by a tool like [CPAN.pm][cpanpm] or [cpanm][cpanm] to install the distribution and its prerequisites. This is the primary purpose of the tools, as something users interact with to test and install modules.

But since authors always had the install tool at hand, it became a convenient place to put author tools as well. The primary of these would be `make dist` which generates a distribution archive suitable for uploading to [CPAN][cpan]. For a long time, this was the primary way authors would create releases.

However, as an author tool, [ExtUtils::MakeMaker][eumm] leaves a lot to be desired. It uses a `MANIFEST` file to list the files to include in the distribution, which won't include new files unless the author remembers to run `make manifest`. And there are other sanity checks that authors should do before making a release that it doesn't provide for at all. Bumping version numbers, updating change logs, tagging and pushing to a VCS server. Some authors would build these tools into their `Makefile.PL` script, but that can become a maintenance burden. Other tools, like [Module::Install][mi] were created to address some of this, by having a split between the author tools and user code, but it made some other problematic architectural choices.

Dist::Zilla
===========

These issues led to the creation of tools like [Dist::Zilla][dzil] which entirely separate the author tooling from the user tooling. It would also take over writing the `Makefile.PL` script, since writing a proper `Makefile.PL` is not trivial, and being able to include arbitrary code is usually unneeded. It is based on a plugin system, allowing authors to tailor its behavior to whatever release process they prefer.

[Dist::Zilla][dzil] can be a fantastic tool, but comes with its own problems. There can be unexpected interactions between the multitude of plugins and versions, which can make the results of builds hard to reproduce without significant effort. It can also be difficult to express the desired build process via the plugin system, or to understand how it will behave by reading its configuration. The combination of these plugins often leads to a significant number of prerequisites, increasing the complexity and avenues of failure. It also regularly gets updated to take advantage of new perl features, which prevents it from being used on older perl versions[^1]. These issues are all trade-offs made to allow it to be very feature full and extremely flexible.

[^1]: It prevents authors from using older perl versions, but the CPAN releases it creates are still backward compatible.

Distar
======

Not all authors agree with the trade-offs made by tools like [Dist::Zilla][dzil]. [Matt S Trout][mst] wanted an author tool, but started from a different perspective. What needs to be added on top of what [ExtUtils::MakeMaker][eumm] provides to make it a viable author tool? The result of this thought process was [Distar][distar].

[Distar][distar] is a tool that adds just enough to [ExtUtils::MakeMaker][eumm] so you can't bungle a release. And it is small enough to be able to read and understand the entire codebase at the same time.

It uses an allowlist to include files in the dist, rather than [ExtUtils::MakeMaker][eumm]'s normal denylist, preventing extraneous files in the working directory from being shipped. It makes certain that the `MANIFEST` is up to date. It ensures you are testing against the same thing you are releasing. It forces you to add a Changelog entry for your release. It uploads to [CPAN][cpan], and tags and pushes to git. And it does very little else.

Since its creation, [Distar][distar] has seen very little adoption. The only other [CPAN][cpan] author using it that I'm aware of is me. I've also done most of the development for the last several years, making it more robust, and including a few quality of life improvements like version bumping.

[Distar][distar] does not make any changes from files in the repository vs the files shipped in the distribution. It generates a `README`, a `LICENSE` file, and the default files created by [ExtUtils::MakeMaker][eumm], `META.json` and `META.yml`.

How Does Distar Work?
=====================

To use [Distar][distar], you start with a normal `Makefile.PL`:

    use ExtUtils::MakeMaker;
    WriteMakefile(NAME => "My::Cool::Module");

Before the call to `WriteMakefile`, you add:

    (do './maint/Makefile.PL.include' or die $@) unless -f 'META.yml';

Since the `META.yml` file is generated by [ExtUtils::MakeMaker][eumm] in the distribution directory it creates, but not the working directory, it is a reliable check for "author mode" in a pure [EUMM][eumm] environment.

You then create the file `maint/Makefile.PL.include` with the content:

    BEGIN { -e 'Distar' or system qw(git clone https://github.com/p5sagit/Distar.git) }
    use lib 'Distar/lib';
    use Distar;

    author 'A. U. Thor <author@cpan.org>';

[Distar][distar] provides essentially no configuration aside from this. Any customization needed in the build process needs to be written in the `Makefile.PL`, the same way it would be for raw [ExtUtils::MakeMaker][eumm].

To make a release using [Distar][distar], the process is:

    make bump           # bump version number
    make nextrelease    # add changelog version heading
    make release        # test & release

make release
------------

The core of [Distar][distar] is the `release` target. It does a number of sanity checks, then creates a release.

  * Verify that the version numbers in all of the modules are matching.
  * Verify that the `MANIFEST` file is up to date.
  * Verify that `cpan-upload` is available to push to CPAN.
  * Fetches any changes from the remote git server
  * Verify that no git tag exists for the version being published.
  * Verify that the release is being made from the master branch.
  * Verify that there are no new commits on the remote git server.
  * Verify that there is a version line in the Changelog that has been staged in git.
  * Verify that there are no other uncommitted changes.
  * Generate a dist directory, including a `README` and `LICENSE` file.
  * Run the tests in the dist directory, with `RELEASE_TESTING` set.
  * Create a git commit for the release.
  * Create a git tag for the release.
  * Create a dist tarball.
  * Upload the dist tarball to [CPAN][cpan].
  * Push the new tag and commit to the remote git server.

make bump
---------

[Distar][distar] provides a version bumping tool to modify any modules or `Makefile.PL` containing the version number. It is meant to work with both numeric and dotted-decimal style versions. By default, it will bump the last segment of the dotted-decimal version, or the highest precision portion of a numeric version. It also provides other targets to bump specific parts of a version. When bumping a specific part of a version, numeric versions will be treated as `X.YYYZZZ` versions, the same way perl versions work. Any components following the one specified will be reset to 0.

    make bumpmajor                # bump the first digit component
    make bumpminor                # bump the second digit component
    make bumpbugfix               # bump the third digit component

Other options can be used via the `V` option.

    make bump V=1.200             # bump to specific version
    make bump V=--alpha           # bump last component, but add an underscore
                                  # to indicate a developer release
    make bump V='major --alpha'   # bump first digit component, but also add
                                  # an underscore.

After bumping the version, the changes will be committed to git. This will work even if there are other changes in the working directory, leaving those changes uncommitted. The `Makefile.PL` will then be re-run pick up the new version.

make nextrelease
----------------

The release process of a [Distar][distar] dist requires you to add a version line to your Changelog file, and stage that change (git add). The `nextrelease` target will do this for you.

It adds the version line, then uses git's interactive add on it, so you will need to confirm the changes. The diff shown should include all of the changelog entries written for the release.

git clone Distar
================

The current way [Distar][distar] is distributed is by adding code to `Makefile.PL` which clones the repository and uses that. Some might think this would present a compatibility or stability issue, but the authors of [Distar][distar] are very committed to maintaining stability and compatibility. Users have to trust that the authors will always keep the code in git in a working state. This trust has been easy to maintain so far, since the known users of [Distar][distar] are its authors.

There may be other users, since its code is small enough that others could understand it without needing any discussion with [Matt S Trout][mst] or myself.

Not being published on [CPAN][cpan] limits the visibility of [Distar][distar]. I have, at times, thought of this as a feature. I've been uncertain if I wanted the pressure of additional users, and having to either accomodate or push back against alternate use cases and feature requests.

Should You Use Distar?
======================

I have no idea. It works well for me, and fits the model I have for creating releases. It isn't as fully automatic as many [Dist::Zilla][dzil] setups are, but it is extremely predictable, backwards compatible, and lightweight. It is very opinionated. It is unlikely to become more configurable to cover differing use cases.

If others did like the idea of [Distar][distar], it may be time to look at improving the documentation and publishing it on CPAN.

[Distar][distar] is currently available on GitHub at https://github.com/p5sagit/Distar.

[cpan]: https://www.cpan.org/
[eumm]: https://metacpan.org/pod/ExtUtils::MakeMaker
[mb]: https://metacpan.org/pod/Module::Build
[cpanpm]: https://metacpan.org/pod/CPAN
[cpanm]: https://metacpan.org/pod/cpanm
[mi]: https://metacpan.org/pod/Module::Install
[dzil]: https://metacpan.org/pod/Dist::Zilla
[mst]: https://metacpan.org/author/MSTROUT
[distar]: https://github.com/p5sagit/Distar