Python package guidelines

Arch package guidelines
32-bit
–
CLR
–
CMake
–
Cross
–
DKMS
–
Eclipse
–
Electron
–
Font
–
Free Pascal
–
GNOME
–
Go
–
Haskell
–
Java
–
KDE
–
Kernel modules
–
Lisp
–
Meson
–
MinGW
–
Node.js
–
Nonfree
–
OCaml
–
Perl
–
PHP
–
Python
–
R
–
Ruby
–
Rust
-
Security
–
Shell
–
VCS
–
Web
–
Wine
This document covers standards and guidelines on writing
PKGBUILDs
for
Python
software.
Package naming
For
Python 3
library modules, use
python-
modulename
. Also use the prefix if the package provides a program that is strongly coupled to the Python ecosystem (e.g.
pip
or
tox
). For other applications, use only the program name.
Note
The package name should be entirely lowercase.
Architecture
See
PKGBUILD#arch
.
A Python package that contains C extensions is architecture-dependent. Otherwise it is most likely architecture-independent.
Packages built using
setuptools
define their C extensions using the
ext_modules
keyword in
setup.py
.
Source
Note
With
RFC0020
the default is to use upstream provided source tarballs, instead of PyPI provided sdist tarballs.
Download URLs linked from the PyPI website include an unpredictable hash that needs to be fetched from the PyPI website each time a package must be updated. This makes them unsuitable for use in a PKGBUILD. PyPI
provides
an alternative stable scheme:
PKGBUILD#source
source=()
array should use the following URL templates:
Source package
https://files.pythonhosted.org/packages/source/${_name::1}/${_name//-/_}/${_name//-/_}-$pkgver.tar.gz
Pure Python wheel package
https://files.pythonhosted.org/packages/py2.py3/${_name::1}/$_name/${_name//-/_}-$pkgver-py2.py3-none-any.whl
(Bilingual – Python 2 and Python 3 compatible)
https://files.pythonhosted.org/packages/py3/${_name::1}/$_name/${_name//-/_}-$pkgver-py3-none-any.whl
(Python 3 only)
Note that the distribution name can contain dashes, while its representation in a wheel filename cannot (they are converted to underscores).
Architecture specific wheel package
Additional architecture-specific arrays can be added by appending an underscore and the architecture name, e.g.
source_x86_64=('...')
. Also
_py=cp310
can be used to not repeat the Python version:
https://files.pythonhosted.org/packages/$_py/${_name::1}/$_name/${_name//-/_}-$pkgver-$_py-${_py}m-manylinux1_x86_64.whl
Note that a custom
_name
variable is used instead of
pkgname
since Python packages are generally prefixed with
python-
. This variable can generically be defined as follows:
_name=${pkgname#python-}
Installation methods
Python packages are generally installed using language-specific package manager such as
pip
, which fetches packages from an online repository (usually
PyPI
, the Python Package Index) and tracks the relevant files.
However, for managing Python packages from within
PKGBUILD
s, one needs to "install" the Python package to the temporary location
$pkgdir
/usr/lib/python
<Python version>
/site-packages/
$pkgname
.
For Python packages using
standard metadata
to specify their build backend in
pyproject.toml
, this can most easily achieved using
python-build
and
python-installer
.
Old packages might fail to specify that they use setuptools, and only offer a
setup.py
that has to be invoked manually.
Note
Dependencies from the package's metadata must be defined in the
depends
array otherwise they will not be installed.
Standards based (PEP 517)
Tip
When building from upstream provided source tarballs and upstream relies on git to derive a version string for the project, it is required to set tooling specific environment variables to
$pkgver
before building a wheel:
python-flit-core
,
python-hatch-vcs
and
python-setuptools-scm
:
SETUPTOOLS_SCM_PRETEND_VERSION
python-pbr
:
PBR_VERSION
python-pdm-backend
:
PDM_BUILD_SCM_VERSION
A standards based workflow is straightforward: Build a wheel using
python-build
and install it to
$pkgdir
using
python-installer
:
Note
Besides
python-build
and
python-installer
, you will also need to add the
build backend
used by the package to
makedepends
. All build backends that are available in the repo are members of the
python-build-backend
group. Check the
build-system.build-backend
in the project's
pyproject.toml
which is applicable for your project. If it is not configured, it defaults to
python-setuptools
.
makedepends=(python-build python-installer python-wheel)

build() {
    cd $_name-$pkgver
    python -m build --wheel --no-isolation
}

package() {
    cd $_name-$pkgver
    python -m installer --destdir="$pkgdir" dist/*.whl
}
where:
--wheel
results in only a wheel file to be built, no source distribution.
--no-isolation
means that the package is built using what is installed on your system (which includes packages you specified in
depends
), by default the tool creates an isolated virtual environment and performs the build there.
--destdir="$pkgdir"
prevents trying to directly install in the host system instead of inside the package file, which would result in a permission error
--compile-bytecode=...
or
--no-compile-bytecode
can be passed to
installer
, but the default is sensibly picked, so this should not be necessary.
Note
Skipping
build
and putting the
.whl
file in your
source
array is discouraged in favor of building from source, and should only be used when the latter is not a viable option (for example, packages which
only
come with wheel sources, and therefore cannot be built from source).
Tip
If your package is a
VCS package
(
python-…-git
), include the command
git -C "${srcdir}/${pkgname}" clean -dfx
in your
prepare
function. This removes stale wheels along with other build artifacts, and helps prevent issues further down the road. See also upstream issues for
setuptools
and
Poetry
.
setuptools or distutils
If no
pyproject.toml
can be found or it fails to contain a
[build-system]
table, it means the project is using the old legacy format, where the project provides a
setup.py
file, which invokes the
setup
function from
setuptools
or
distutils.core
.
Such packages usually can also be built and installed using the method described above using
python-build
and
python-installer
, and this is the preferred method. But they will also additionally need
python-setuptools
in
makedepends
.
You
can
still build and install these packages using the old, deprecated way of running
setup.py
directly, which is shown below and it is described here for those cases, where the pep-517 compliant way does not work for some reason.
But note that you will get the following warning in the output of the
package
step, when building a package using this method:
SetuptoolsDeprecationWarning: setup.py install is deprecated.
Also note that Python versions from 3.12 onwards do not include
distutils
in the standard library any more, which means that packages for projects still using
setup.py
generally need to have
python-setuptools
in
makedepends
, since that provides its own version of
distutils
.
makedepends=('python-setuptools')

build() {
    cd $_name-$pkgver
    python setup.py build
}

package() {
    cd $_name-$pkgver
    python setup.py install --root="$pkgdir" --optimize=1
}
where:
--root="$pkgdir"
works like
--destdir
above
--optimize=1
compiles optimized bytecode files (
.opt-1.pyc
) so they can be tracked by
pacman
instead of being created on the host system on demand.
Adding
--skip-build
optimizes away the unnecessary attempt to re-run the build steps already run in the
build()
function, if that is the case.
If a package uses
python-setuptools-scm
, the package most likely will not build with an error such as:
LookupError: setuptools-scm was unable to detect version for /build/python-jsonschema/src/jsonschema-3.2.0.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
To get it building
SETUPTOOLS_SCM_PRETEND_VERSION
has to be exported as an environment variable with
$pkgver
as the value:
export SETUPTOOLS_SCM_PRETEND_VERSION=$pkgver
Check
Note
Avoid using
tox
to run testsuites as it is explicitly designed to test repeatable configurations downloaded from PyPI while
tox
is running, and does
not
test the version that will be installed by the package. This defeats the purpose of having a
check
function at all.
Avoid adding pytest plugins for linting, coverage or type checking to
checkdepends
(see the
#Disable pytest options
section for details). This makes bootstrapping harder and these tests are not required for distribution packaging as they do not test functionality.
Avoid using relative paths when setting a custom
PYTHONPATH
for tests, as it may lead to non-obvious error cases. Use absolute paths instead.
Most Python projects providing a testsuite use the unittest runner or nosetests or pytest (provided by
python
and
python-nose
and
python-pytest
, respectively) to run tests with
test
in the name of the file or directory containing the testsuite. In general, simply running
nosetests
or
pytest
is enough to run the testsuite.
check(){
    cd $_name-$pkgver

    # Builtin unittest
    python -m unittest discover -v

    # For nosetests
    nosetests

    # For pytest
    pytest
}
If there is a compiled C extension, the tests need to be run using a
$PYTHONPATH
, that reflects the current major and minor version of Python in order to find and load it.
check(){
    cd $_name-$pkgver
    local python_version=$(python -c 'import sys; print("".join(map(str, sys.version_info[:2])))')

    # Builtin unittest
    PYTHONPATH="$PWD/build/lib.linux-$CARCH-cpython-$python_version" python -m unittest discover -vs .

    # For nosetests
    PYTHONPATH="$PWD/build/lib.linux-$CARCH-cpython-$python_version" nosetests

    # For pytest
    PYTHONPATH="$PWD/build/lib.linux-$CARCH-cpython-$python_version" pytest
}
Tips and tricks
Using Python version
Sometimes during preparing, building, testing or installation it is required to refer to the system's major and minor Python version (e.g.
3.9
or
3.10
). Do not hardcode this and instead use a call to the Python interpreter to retrieve the information and store it in a local variable:
check(){
  local python_version=$(python -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')
  ...
}
Using site-packages
Sometimes during building, testing or installation it is required to refer to the system's
site-packages
directory. Do not hardcode this directory and use a call to the Python interpreter instead to retrieve the path and store it in a local variable:
check(){
  local site_packages=$(python -c "import site; print(site.getsitepackages()[0])")
  ...
}
Test directory in site-package
Make sure to not install a directory named just
tests/
directly under
site-packages/
(i.e.
/usr/lib/pythonX.Y/site-packages/tests/
). Doing so could lead to conflicts between packages. Python package projects using
setuptools
are sometimes misconfigured to include the directory containing its tests as a top level Python package. If you encounter this, you can help by filing an issue with the package project and ask them to fix this, e.g.
like this
.
Disable pytest options
When running pytest, it is mostly desirable to not run with additional plugins. Especially plugins for linting and coverage are counterproductive in packaging, as changes in behavior may have tests fail.
To disable pytest options such as
addopts
it is preferred to use an option override on the command line over patching any configuration files used by pytest due to the maintenance overhead.
To unset all additional options use
pytest -o addopts=""
Fix reproducibility issue with meson-python
When using
meson-python
as a PEP 517 build backend it uses randomized folder paths
that create reproducibility issues
. This can be circumvented by hardcoding the used folder with the
-Cbuild-dir
flag:
python -m build --wheel --no-isolation -Cbuild-dir=build
Running tests with an installed package
Some packages require being installed to have the tests run successfully. In such cases, such as for example
python-narwhals
, you can create a
virtual environment
to install the built package and run the tests there:
python -m venv --system-site-packages test-env
test-env/bin/python -m installer dist/*.whl
test-env/bin/python -P -m pytest