jarn.mkrelease 5.3
Python package releaser
License: BSD-2-Clause
Requires Python: !=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7
Keywords: create,release,upload,python,sdist,sdists,wheel,wheels,pypi,releaser,package,packages,publish

==============
jarn.mkrelease
==============
---------------------------------------------------
Python package releaser
---------------------------------------------------

**mkrelease** is a no-frills Python package releaser. It is designed to take
the cumber out of building and distributing Python packages.
The releaser supports source distributions, eggs, and wheels.

Motivation
==========

After preparing a package for release (update version strings, dates) we
typically have to:

1. Commit modified files.

2. Tag the release.

3. Build a source distribution and wheel.

4. Distribute the results via scp or upload them to an index server.

If we are doing this a lot, the need for automation becomes obvious.

Contents
========

* Installation_
* Usage_
* Options_
* Arguments_
* Configuration_
* `Upload with SCP`_
* `Upload to Index Servers`_
* `Using GnuPG`_
* Requirements_
* Footnotes_
* Related_
* Changelog_

Installation
============

mkrelease works with Python 2.7 - 3.11 and all released versions of
setuptools.

Use ``pip install jarn.mkrelease`` to install the ``mkrelease`` script.

Since version 4.4 mkrelease requires twine_ for register and upload
operations. Twine may be installed as a global utility on the system PATH or
into the same environment as jarn.mkrelease. [1]_

Use ``pip install jarn.mkrelease[twine]`` to install mkrelease + twine.

Since version 5.0 file-finder extensions are no longer installed by default.
If you still want to use file-finders specify the ``filefinder`` extra.

Use ``pip install jarn.mkrelease[filefinder]`` to install mkrelease + file-finder
extensions.

Use ``pip uninstall setuptools-subversion setuptools-hg setuptools-git`` to
uninstall file-finder extensions.

.. _twine: https://twine.readthedocs.io/en/stable/

Usage
=====

``mkrelease [options] [scm-sandbox|scm-url [rev]]``

Options
=======

``-C, --no-commit``
    Do not commit modified files from the sandbox.

``-T, --no-tag``
    Do not tag the release in SCM.

``-P, --no-push``
    Do not push commits and tags upstream.

``-R, --no-register``
    Do not register the release with dist-location.

``-S, --no-upload``
    Do not upload the release to dist-location.

``-n, --dry-run``
    Dry-run; equivalent to ``-CTPRS``. This flag turns mkrelease into a
    simple package builder.

``--svn, --hg, --git``
    Select the SCM type. Only required if the SCM type
    cannot be guessed from the argument.

``-d dist-location, --dist-location=dist-location``
    An scp destination specification, an index
    server configured in ``~/.pypirc``, or an alias name for
    either. This option may be specified more than once.

``-s, --sign``
    Sign the release with GnuPG.

``-i identity, --identity=identity``
    The GnuPG identity to sign with. Implies ``-s``.

``-z, --zip``
    Release a zip archive.

``-g, --gztar``
    Release a tar.gz archive (default).

``-b, --egg``
    Release a binary egg.

``-w, --wheel``
    Release a wheel file (default).

``-m, --manifest-only``
    Ignore setuptools file-finder extensions and collect files via
    ``MANIFEST.in`` only.

``-e, --develop``
    Allow setuptools build tags. Implies ``-T``.

``-q, --quiet``
    Suppress output of setuptools commands.

``-t twine, --twine=twine``
    Override the twine executable used.

``-c config-file, --config-file=config-file``
    Use config-file instead of the default ``~/.mkrelease``.

``-l, --list-locations``
    List known dist-locations and exit.

``-h, --help``
    Print the help message and exit.

``-v, --version``
    Print the version string and exit.

``--no-color``
    Disable output colors.

``--non-interactive``
    Do not prompt for username and password if the
    required credentials are missing.

Arguments
=========

``scm-sandbox``
    A local SCM sandbox. Defaults to the current working
    directory.

``scm-url [rev]``
    The URL of a remote SCM repository. The optional ``rev``
    argument specifies a branch or tag to check out.

Configuration
=============

mkrelease reads available index servers from the distutils_ configuration
file ``~/.pypirc``. This file should contain your PyPI account information: [2]_

.. code:: ini

  [distutils]
  index-servers =
      pypi

  [pypi]
  repository = https://upload.pypi.org/legacy/
  username = fred
  password = secret

Next, mkrelease reads its own configuration file ``~/.mkrelease``.
It may be used to change defaults and to define location
aliases:

.. code:: ini

  [mkrelease]
  # Release steps
  commit = yes
  tag = yes
  push = yes
  register = no
  upload = yes

  # One or more of: zip gztar egg wheel
  formats = gztar wheel

  # Setuptools options
  manifest-only = yes
  develop = no
  quiet = no

  # Sign with GnuPG
  sign = no
  identity =

  # Default dist-location
  dist-location =

  [aliases]
  # Map name to one or more dist-locations
  customerA =
      jarn.com:/var/dist/customerA/
  public =
      jarn.com:/var/dist/public/
  world =
      pypi
      public

.. _distutils: https://packaging.python.org/en/latest/specifications/pypirc/

Upload with SCP
===============

The simplest distribution location is a server directory reachable by ssh.
Releasing a package means scp-ing it to the appropriate place
on the server:

.. code::

  $ mkrelease -d customerA
  $ mkrelease -d jarn.com:/var/dist/customerB/
  $ mkrelease -d scp://jarn.com/var/dist/customerC/
  $ mkrelease -d stefan@jarn.com:eggs -C -e -q

To upload via sftp instead of scp, use the ``sftp`` URL scheme:

.. code::

  $ mkrelease -d sftp://jarn.com/var/dist/customerD/

Note: Unlike scp, the sftp client does not prompt for login credentials.
This means that non-interactive login must be configured on the
destination server or the upload will fail.

Upload to Index Servers
=======================

Another way of publishing a Python package is by uploading it to a dedicated
index server like PyPI_.
Given the ``~/.pypirc``
file from above, we can release to PyPI simply by typing:

.. code::

  $ mkrelease -d pypi

Index servers are not limited to PyPI though.
There is TestPyPI_, and there are alternative index servers like
devpi_.
We extend our ``~/.pypirc``: [2]_

.. code:: ini

  [distutils]
  index-servers =
      pypi
      testpypi

  [pypi]
  repository = https://upload.pypi.org/legacy/
  username = fred
  password = secret

  [testpypi]
  repository = https://test.pypi.org/legacy/
  username = fred
  password = secret

We can now release to TestPyPI with:

.. code::

  $ mkrelease -d testpypi -C -e

.. _PyPI: https://pypi.org/
.. _TestPyPI: https://test.pypi.org/
.. _devpi: https://www.devpi.net

Using GnuPG
===========

Release a package and sign the distributions with GnuPG:

.. code::

  $ mkrelease -d pypi -s -i fred@bedrock.com

The ``-i`` flag is optional and GnuPG will pick your default
key if not given.

Requirements
============

The following commands must be available on the system PATH (you only need
what you plan to use):

* svn

* hg

* git

* scp

* sftp

* gpg

* twine [1]_

Footnotes
=========

.. [1] The twine executable is determined by trying in order:

    1. Value of ``--twine`` command line option, or
    2. Value of ``TWINE`` environment variable, or
    3. Value of ``twine`` configuration file setting, or
    4. ``python -m twine`` if twine is importable, or
    5. ``twine``

.. [2] There are more secure ways to handle login credentials:

    1. Generate an `API token`_ on PyPI, and/or
    2. Make use of twine's `keyring support`_.

.. _`API token`: https://pypi.org/help/#apitoken
.. _`keyring support`: https://twine.readthedocs.io/en/stable/index.html#keyring-support

Related
=======

Also see our Python documentation viewer `jarn.viewdoc`_.

.. _`jarn.viewdoc`: https://github.com/Jarn/jarn.viewdoc


Changelog
=========

5.3 - 2023-09-23
----------------

- Update tox.ini for latest tox.
  [stefan]

- Filter deprecated config warning.
  [stefan]

- Fix invalid escape sequences.
  [stefan]

- The ``rev`` part of Git URLs no longer defaults to 'master'.
  [stefan]

5.2 - 2023-01-20
----------------

- Support setuptools >= 65.6.0.
  [stefan]

5.1 - 2022-11-13
----------------

- Support Python 2.7 - 3.11.
  [stefan]

- Do not require setup.py or setup.cfg if pyproject.toml exists.
  [stefan]

- Filter some deprecation warnings.
  [stefan]

- Disable twine progress bars in non-interactive mode.
  [stefan]

5.0 - 2022-02-26
----------------

- NEW: File finder extensions are no longer installed by default.
  [stefan]

- NEW: push is now on by default
  [stefan]

- NEW: register is now off by default
  [stefan]

- NEW: formats now defaults to gztar + wheel
  [stefan]

- Do not require setup.py if setup.cfg exists.
  [stefan]

- Switch to the nose2 test runner for its fast multiprocessing plugin.
  [stefan]

4.4 - 2022-02-26
----------------

- Support Python 2.7 - 3.10.
  [stefan]

- Support local distutils in setuptools >= 60.0.0.
  [stefan]

- Add output colors.
  [stefan]

- Require twine_ for register and upload operations.
  [stefan]

- Move metadata to setup.cfg.
  [stefan]

- Move tests out of ``jarn.mkrelease`` namespace.
  [stefan]

- Include tests in sdist but not in wheel.
  [stefan]

4.3 - 2019-01-28
----------------

- Fix issue #10: Can no longer run from a zc.buildout.
  [stefan]

- Support ``python -m jarn.mkrelease``.
  [stefan]

4.2 - 2019-01-25
----------------

- Drop Python 2.6 support, add Python 3.7.
  [stefan]

- Update wheel and keyring dependencies.
  [stefan]

- Add a complete config file example to the README.
  [stefan]

- Accept ``dist-location`` as alias for ``distdefault`` in config files.
  [stefan]

- Convert dashes to underscores in config parser optionxform.
  [stefan]

4.1 - 2017-10-06
----------------

- Add -m option to skip setuptools extensions.
  [stefan]

- Never read existing ``SOURCES.txt`` files.
  [stefan]

- Allow --egg as alias for --binary.
  [stefan]

4.0.1 - 2017-07-20
------------------

- Fix conditional include of keyring.
  [stefan]

4.0 - 2017-07-20
----------------

- Add support for source distributions in gztar format.
  [stefan]

- Add support for wheel files.
  [stefan]

- Allow multiple archive formats per release.
  [stefan]

- Protect against bad or incomplete locale settings.
  [stefan]

- The secondary thread would sometimes not terminate in Python 2.
  [stefan]

3.10 - 2017-02-01
-----------------

- Support setuptools >= 33.1.0.
  [stefan]

3.9 - 2017-01-31
----------------

- Catch up with late-2016 PyPI API change.
  [stefan]

- Add -R option to skip the register step.
  [stefan]

- Support Python 2.6 - 3.6 without 2to3.
  [stefan]

- Handle return code changes in Mercurial 3.1.1.
  [stefan]

3.8 - 2013-11-21
----------------

- Support Python 3.x.
  [stefan]

3.7 - 2012-08-22
----------------

- Fix compilation of Python source files when the -b option is given.
  [stefan]

- Run check command as part of sdist and register commands.
  [stefan]

- Add SFTP support.
  [stefan]

- Allow ``sftp://`` and ``scp://`` URLs as dist-locations.
  [stefan]

3.6 - 2012-07-11
----------------

- Handle return code changes in Mercurial 2.1.
  [stefan]

- Add setuptools-subversion dependency.
  [stefan]

- Support Subversion 1.7 with the help of setuptools-subversion.
  [stefan]

3.5 - 2011-11-25
----------------

- Allow multiple values for the ``distdefault`` config file option.
  [stefan]

- Defer list-locations until after all arguments have been parsed.
  [stefan]

- Make tests run twice as fast by avoiding Subversion checkouts.
  [stefan]

3.4 - 2011-11-10
----------------

- Warn if -p is given but no upstream location is found.
  [stefan]

- Always push to default in Mercurial.
  [stefan]

- Avoid reading empty lines from terminating subprocesses.
  [stefan]

- Fix bug in handling of distbase.
  [stefan]

3.3 - 2011-10-31
----------------

- Add setuptools to the PYTHONPATH for subprocesses.
  [stefan]

- Unset any PYTHONPATH while executing SCM commands.
  [stefan]

- Support Git's short-form ``ssh://`` URLs.
  [stefan]

- Add -c option to specify a config file other than ~/.mkrelease.
  [stefan]

3.2 - 2011-10-21
----------------

- Fix the environment passed to subprocesses; Mercurial did not appreciate
  the mangled PYTHONPATH.
  [stefan]

- Allow to specify the branch or tag to check out from Git and Mercurial
  repositories.
  [stefan]

- Adapt to new status output in Subversion 1.6.
  [stefan]

- Always include ``distdefault`` in list-locations.
  [stefan]

- Detect Subversion repos from ``file://`` URLs.
  [stefan]

- Detect bare Git repos from ``file://`` URLs.
  [stefan]

3.1 - 2011-07-19
----------------

- Pass the PYTHONPATH to subprocesses so mkrelease works in zc.buildout
  environments.
  [stefan]

- Improve SCM detection in situations where one or more SCMs are nested.
  [stefan]

- Add support for relative ``file:`` URLs.
  [stefan]

- Depend on lazy_ instead of carrying a local implementation.
  [stefan]

.. _lazy: https://github.com/stefanholek/lazy

3.0.10 - 2011-07-07
-------------------

- Add -l option to list known dist-locations (i.e. servers and aliases).
  [stefan]

- Drop support for server URLs as dist-locations. Server URLs are
  not unique.
  [stefan]

- Update the Mercurial test repository so tagging tests don't fail
  under Mercurial 1.8.
  [stefan]

3.0.9 - 2010-12-31
------------------

- Rename ``[defaults]`` configuration file section to ``[mkrelease]``.
  [stefan]

- Various internal code cleanups.
  [stefan]

3.0.8 - 2010-08-13
------------------

- Avoid underscores in dependency names.
  [stefan]

- Handle return code changes in Mercurial 1.6.
  [stefan]

3.0.7 - 2010-07-07
------------------

- Improve documentation and error messages.
  [stefan]

3.0.5 - 2010-03-23
------------------

- Allow per-server configuration of -s and -i defaults.
  [stefan]

- Support the codespeak.net Subversion repository layout.
  [stefan]

3.0.4 - 2010-03-16
------------------

- Status checks didn't use the same path restrictions as commits
  (Mercurial and Git.)
  [stefan]

3.0.3 - 2010-03-16
------------------

- Change how we check for existing tags in Subversion repositories.
  [stefan]

- Make sandbox-status checks more robust in all three SCMs.
  [stefan]

3.0.2 - 2010-03-12
------------------

- Add support for Git 1.7.
  [stefan]

3.0.1 - 2010-02-07
------------------

- Stop when -d pypi is given but no configuration can be found.
  [stefan]

- Use ``gnu_getopt`` to parse the command line.
  [stefan]

3.0 - 2010-01-15
----------------

- Switch to -n for dry-run to be consistent with other tools.
  [stefan]

- Rename --skip-* long options to --no-* for the same reason.
  [stefan]

- Fix a bug in Mercurial and Git sandbox detection.
  [stefan]

- Prepare for standalone distutils.
  [stefan]

2.0.4 - 2010-01-10
------------------

- Improve Git support to handle remotes other than origin.
  [stefan]

- Fix SCM detection in ``ssh://`` URLs.
  [stefan]

2.0.3 - 2010-01-03
------------------

- Add -b option for releasing binary eggs.
  [stefan]

- Don't choke on dirty sandboxes when dry-running.
  [stefan]

2.0.2 - 2009-08-29
------------------

- Filter meta files (``.svn*``, ``.hg*``, ``.git*``) and never include
  them in releases.
  [stefan]

- Make sure to clean up all temporary files.
  [stefan]

2.0.1 - 2009-07-24
------------------

- Fixed bug which could cause mkrelease to issue eggs with faulty manifest
  files (Symptom: data files not installed).
  [stefan]

- The -e flag now implies -T. We never want to tag a development release.
  [stefan]

2.0 - 2009-07-16
----------------

- Allow command line options to appear after the argument. As in:
  ``mkrelease src/mypackage -q -d pypi``.
  [stefan]

2.0b2 - 2009-07-09
------------------

- Improve user feedback in the SCM-detection part.
  [stefan]

- Document the -e flag.
  [stefan]

- Drop global configuration file for YAGNI.
  [stefan]

- Allow to set default values for -s and -i in ~/.mkrelease.
  [stefan]

2.0b1 - 2009-07-03
------------------

- By default, ignore all version number extensions (dev-r12345)
  that may be configured in setup.cfg. Passing the -e flag
  disables this safeguard.
  [witsch, stefan]

- Delete any existing signature file before signing anew. This keeps
  GnuPG from complaining about existing (but left-over) files.
  [stefan]

2.0a2 - 2009-06-27
------------------

- Drop configurable Python and use sys.executable. This also means we
  now require Python 2.6.
  [stefan]

- Force setuptools to only use file-finders for the selected SCM type.
  This is required to support multi-SCM sandboxes (think git-svn).
  [stefan]

- Treat Subversion sandboxes just like the others and avoid the
  temporary checkout step.
  [stefan]

- Remove the -u flag for being pointless.
  [stefan]

2.0a1 - 2009-06-14
------------------

- Added support for Mercurial and Git.
  [stefan]

- Added 250+ unit tests.
  [stefan]

1.0.2 - 2009-06-13
------------------

- Documented long options.
  [stefan]

- Print a "Tagging ..." line before tagging.
  [stefan]

1.0 - 2009-05-14
----------------

- Print help and version to stdout, not stderr.
  [stefan]

1.0b4 - 2009-04-30
------------------

- Since distutils commands may return 0, successful or not, we must
  check their output for signs of failure.
  [stefan]

- Allow to pass argument list to ``main()``.
  [stefan]

1.0b3 - 2009-03-23
------------------

- No longer depend on grep.
  [stefan]

- Use subprocess.Popen instead of os.system and os.popen.
  [stefan]

- Protect against infinite alias recursion.
  [stefan]

- Drop -z option and always create zip files from now on.
  [stefan]

1.0b2 - 2009-03-19
------------------

- Checkin everything that's been modified, not just "relevant" files.
  [stefan]

- Expand aliases recursively.
  [stefan]

1.0b1 - 2009-03-18
------------------

- The distbase and distdefault config file options no longer have
  default values.
  [stefan]

- Read index servers from ~/.pypirc and allow them to be used with -d.
  [stefan]

- The -d option may be specified more than once.
  [stefan]

- Dropped -p option. Use -d pypi instead.
  [stefan]

- Dropped -c option. If your have non-standard SVN repositories you must
  tag by hand.
  [stefan]

0.19 - 2009-02-23
-----------------

- Absolute-ize the temp directory path.
  [stefan]

0.18 - 2009-01-26
-----------------

- Include README.txt and CHANGES.txt in long_description.
  [stefan]

- Rid unused imports and locals.
  [stefan]

0.17 - 2009-01-23
-----------------

- Add -c option to enable codespeak support. The codespeak.net repository
  uses ``branch`` and ``tag`` instead of ``branches`` and ``tags``.
  [gotcha, stefan]

0.16 - 2009-01-13
-----------------

- Fold regex construction into find and make find a method.
  [stefan]

- Update README.txt.
  [stefan]

0.15 - 2009-01-13
-----------------

- Support for reading default options from a config file.
  [fschulze, stefan]

0.14 - 2009-01-08
-----------------

- Add -s and -i options for signing PyPI uploads with GnuPG.
  [stefan]

- Stop execution after any failing step.
  [stefan]

0.13 - 2009-01-05
-----------------

- Stop execution when the checkin step fails.
  [stefan]

0.12 - 2009-01-02
-----------------

- setup.cfg may not exist.
  [stefan]

0.11 - 2008-12-02
-----------------

- Add setup.cfg to list of files we check in.
  [stefan]

0.10 - 2008-10-21
-----------------

- Don't capitalize GetOptError messages.
  [stefan]

0.9 - 2008-10-16
----------------

- Add -v option to print the script version.
  [stefan]

0.8 - 2008-10-16
----------------

- Lift restriction where only svn trunk could be released.
  [stefan]

0.7 - 2008-10-09
----------------

- Fix PyPI upload which must happen on the same command line as sdist.
  [stefan]

0.6 - 2008-10-08
----------------

- Update README.txt.
  [stefan]

0.5 - 2008-10-08
----------------

- Also locate and checkin HISTORY.txt to support ZopeSkel'ed eggs.
  [stefan]

0.4 - 2008-10-08
----------------

- Use svn checkout instead of svn export because it makes a difference
  to setuptools.
  [stefan]

- Add -p option for uploading to PyPI instead of dist-location.
  [stefan]

0.3 - 2008-10-06
----------------

- Also locate and checkin version.txt.
  [stefan]

0.2 - 2008-10-01
----------------

- Add -z option to create zip archives instead of the default tar.gz.
  [stefan]

0.1 - 2008-10-01
----------------

- Initial release
  [stefan]