Package: packit
PacKitContents:RationaleOverviewUsageFacilitiesIncluding Files Other than Python LibrariesFurther DevelopmentRationaleCreating python packages is routine operation that involves a lot of
actions that could be automated. Although there are petty good tools
likepbrfor that purpose,
they miss some features and lack flexibility by trying to enforce some
strongly opinionated decisions upon you. PacKit tries to solve this by
providing a simple, convenient, and flexible way to create and build
packages while aiming for following goals:simple declarative way to configure your package throughsetup.cfgfollowingdistutils2 setup.cfg
syntaxreasonable defaultsopen for extensionOverviewPacKit is wrapper aroundpbrthough it only uses it for interaction with setuptools/distutils through
simplified interface. None ofpbrfunctions are exposed
but instead PacKit provides its own interface.Available facilitiesHere's a brief overview of currently implemented facilities and the list
will be extended as new ones will be added.auto-version- set package version depending on selected
versioning strategy.auto-description- set package long descriptionauto-license- include license file into distributionauto-dependencies- populateinstall_requiresandtest_requiresfrom requirement filesauto-packages- discover packages to include in distribution.auto-extra-meta- add useful options to the metadata config
sectionauto-package-data- include all files tracked bygitfrom
package dirs only.auto-tests- makepython setup.py testrun tests withtoxorpytest(depending ontox.inipresence).On top of that PacKit forces easy_install to honor followingPIP's
fetch
directives:index_urlfind_linksPlanned facilitiesauto-plate- integration withplatterauto-license- fill out license informationauto-pep8- produce style-check reportsauto-docs- API docs generationauto-clean- configurable clean jobsauto-coverage(?) - produce coverage reports while running testsIf you don't see desired facilities or have cool features in mind feel
free to contact us and tell about your ideas.UsageCreate asetup.pyin your project dir: :from setuptools import setup

setup(setup_requires='packit', packit=True)That was the first and the last time you touched that file for your
project.Now let's create asetup.cfgthat you will use in order to configure
your package:[metadata]
name = cool-packageAnd... if you're not doing anything tricky in your package then that's
enough! And if you do, take a look at the section below.FacilitiesCurrently all available facilities are enabled by default. Though you
can easily turn them off by usingfacilitiessection in yoursetup.cfg:[facilities]
auto-version = 0
auto-dependencies = f
auto-packages = false
auto-package-data = n
auto-tests = noIf facility is explicitly disabled it won't be used even if
facility-specific configuration section is present.Facility-specific defaults and configuration options described below.auto-versionWhen enabled,auto-versionwill generate and set package version
according to selected versioning strategy.Versioning strategy can be selected usingtypefield underauto-versionsection withinsetup.cfg. The default is:[auto-version]
type = git-pep440
output = src/templates/version.htmlYou can useoutputfield to ask PacKit to write generated version
value into specified filename. The specified filename do not need to
exist but the parent directories should exist. Provided path should
always use forward slashes.git-pep440GeneratePEP440-compliant
version fromannotatedgittags. It's expected that you are using
git tags that followpublic version
identifierdescription andgit-pep440will just append number of commits since
tag was applied to your tag value (theNinpublic version
identifierdescription).If number of commits since tag equal to 0 (your building the tagged
version) theNvalue won't be appended. Otherwise, it will be appended
andlocal version
identifierequal to first 7 chars of commit hash will be also added.Please note: you must create anannotatedtag, otherwise it will be
ignored.Example: 1. <git tag -a 1.2.3.dev -m "dev release 1.2.3.dev"> ->
version is1.2.3.dev<git commit> -> version is1.2.3.dev.post1<git commit> -> version is1.2.3.dev.post2<git tag -a 1.2.3.a -m "Release 1.2.3.a"> -> version is1.2.3.a<git commit> -> version is1.2.3.a.post1<git tag -a 1.2.3 -m "Release 1.2.3"> -> version is1.2.3<git commit> -> version is1.2.3.post1<git commit> -> version is1.2.3.post2fixedUse value specified invalue(it's required when this strategy is
used) underauto-versionsection insetup.cfg:[auto-version]
type = fixed
value = 3.3fileRead a line using UTF-8 encoding from the file specified invalue(it's required when this strategy is used) underauto-versionsection
insetup.cfg, strip it and use as a version.[auto-version]
type = file
value = VERSION.txtshellExecute command specified invalue(it's required when this strategy
is used) underauto-versionsection insetup.cfg, read a line fromstdout, strip it and use as a versioncompositeThe most advanced version strategy designed for special cases. It allows
you to generate complex version values based on other version
strategies. The usage is pretty simple though:[auto-version]
type = composite
value = {foo}.{bar}+{git}
output = main.version

[auto-version:foo]
type = fixed
value = 42
output = 1st.version

[auto-version:bar]
type = shell
value = echo $RANDOM

[auto-version:git]
type = git-pep440
output = 3rd.versionThevaluefield in composite version strategy should be a validstring format
expression.Please note thatoutputdirectives used here only for reference (to
show that they can be used anywhere) and are not required.It's OK to define 'extra' version components and not use them but it's
an error to not define any of components mentioned in composite version
template.auto-descriptionWhen enabled will fill outlong_descriptionfor package from a readme.Thereadmefile name could be specified withfilefield underauto-descriptionsection.If no file name provided, it will be discovered automatically by trying
following list of files:READMEreadmeCHANGELOGchangelogEach of these files will be tried with following extensions:<without extension>.md.markdown.mkdn.text.rst.txtThe readme file will be included in the package data.auto-licenseWhen enabled will include the license file into the distribution.The license file name could be specified by thefilefield withinauto-licensesection.If license file name is not provided the facility will try to discover
it in the current dir trying following file names:LICENSElicenseEach of these files will be tried with following extensions:<without extension>.md.markdown.mkdn.text.rst.txtauto-dependenciesWhen enabled will fillinstall_requiresandtest_requiresfrom
requirement files.Requirement files could be specified byinstallandtestfields
under theauto-dependenciessection of thesetup.cfg.If requirements file names not provided then the facility will try to
discover them automatically.For installation requirements following paths will be tried:requiresrequirementsrequirements/prodrequirements/releaserequirements/installrequirements/mainrequirements/baseFor testing requirements following paths will be tried:test-requirestest_requirestest-requirementstest_requirementsrequirements_testrequirements-testrequirements/testFor each path following extensions will be tried<without extension>.pip.txtOnce a file is found, PacKit stops looking for more files.You can use vcs project urls and/or archive urls/pathsas described
inpip
usage-
they will be split in dependency links and package names during package
creation and will be properly handled by pip/easyinstall during
installation. Remember that you can also make "includes" relationships
betweenrequirements.txtfiles by including a line like-r other-requires-file.txt.auto-packagesWhen enabled and no packages provided insetup.cfgthroughpackagesoption underfilessection will try to automatically find out all
packages in current dir recursively.It operates usingexcludeandincludevalues that can be specified
underauto-packagessection withinsetup.cfg.Ifexcludenot provided the following defaults will be used:test,docs,.toxandenv.Ifincludenot provided,auto-packageswill try the following steps
in order to generate it:Ifpackages_rootvalue provided underfilessection insetup.cfg, it will be used.Otherwise the current working dir will be scanned for any python
packages (dirs with __init__.py) while honoring excludevalue.This packages also will be included into the resulting list of
packages.Onceincludevalue is determined, the resulting packages list will be
generated using following algorithm:for path in include:
    found_packages |= set(find_packages(path, exclude))auto-extra-metaWhen enabled, adds a number of additional options to 'metadata' section.Right now, only 1 extra option supported:is_pure- allows you to override 'purity' flag for
distribution, i.e. you can explicitly say whether your distribution
is platform-specific or no.auto-testsHas no additional configuration options [yet].When enabled, thepython setup.py testis equal to running:toxiftox.iniis presentpytestwithpytest-gitignoreandteamcity-messagesplugins enabled by default otherwise (if you need any other plugins
just add them to test requirements) and activate them with
additional options (see below)The facility automatically downloads underlying test framework and
install it - you don't need to worry about it.You can pass additional parameters to the underlying test framework with
'-a' or '--additional-test-args='.auto-package-dataSee the next section.Including Files Other than Python LibrariesOften, you need to include a data file, or another program, or some
other kind of file, with your Python package. Here are a number of
common situations, and how to accomplish them using packit:Placing data files with the code that uses them: auto-package-dataThe default is that theauto-package-datafacility is enabled. In this
configuration, you can include data files for your python library very
easily by just:Placing them inside a Python package directory (so next to an__init__.pyor in a subdirectory), andAdding them to git version control.setup.cfg
src/
src/nicelib/
src/nicelib/__init__.py
src/nicelib/things.py
src/nicelib/somedata.csvNo change in setup.cfg is required.Putting the files here will cause the
packaging system to notice them and install them in the same arrangement next
to your Python files, but inside the virtualenv where your package is
installed.Once this is done, you have several easy options for accessing them, and all of
these should work the same way in development and once installed:The least magical way ispathlib.Path(__file__).parent / 'somedata.csv',
or some equivalent withos.pathcalls.  This makes your package non-zip-safe,
so it can't be used in apexorzipappapplication.The new hotness isimportlib.resources.open_text('nicelib', 'somedata.csv')andrelated
functions,
available in the stdlib in Python 3.7+ or as a backport in theimportlib_resourcesPyPI package.  One limitation is this doesnotsupport putting resources deeper in subdirectories.The previous standard has beenpkg_resources.resource_stream('nicelib', 'somedata.csv')andrelated
functions.
This supports deeper subdirectories, but ismuchslower thanimportlib.resources.  You shouldn't need to installpkg_resources,
it's part ofsetuptools, which is always available these days.You can turn off theauto-package-datafacility if youdon'twant this
file inclusion mechanism to happen:[facilities]
auto-package-data = noauto-package-datawill not work if your Python package is not at the
root of your git repository (setup.pyis not next to.git).Placing data files relative to the virtual environmentYou can also place files relative to the virtualenv, rather than inside
the package hierarchy (which would be invirtualenv/lib/python*/site-packages/something). This is often used
for things like static files in a Django project, so that they are easy
to find for an external web server. The syntax for this is:[files]
data_files =
    dest_dir = src_dir/**
    dest_dir = file_to_put_thereIn this example,dest_dirwill be created within the top level of the
virtualenv. The contents ofsrc_dirwill be placed inside it, along
withfile_to_put_there.If you need to include acompiled executable filein your package, this
is a convenient way to do it - includebin = bin/**for example. See
thefastatoolspackage for an example of this.  There is also aconfluence page with more details on including compiled
programs.Including Python scriptsScripts need to be treated specially, and not just dropped intobinusingdata_files, because Python changes the shebang (#!) line to
match the virtualenv's python interpreter. This means you can directly
run a script without activating a virtualenv - e.g.env/bin/pip install attrswill work even ifenvisn't activated.[1]If you have some scripts already, the easiest thing is to collect them
in one directory, then usescripts:[files]
scripts =
  bin/*Alternatively, setuptools has a special way to directly invoke a Python
function from the command line, called theconsole_scriptsentry
point.pull-sp-subis an internal package that uses this:[entry_points]
console_scripts =
  pull-sp-sub = pull_sp_sub:mainTo explain that last line, it'sname-of-the-script=dotted-path-of-the-python-module:name-of-the-python-function. So
with this configuration, once the package is installed, setuptools
creates a script at$VIRTUAL_ENV/bin/pull-sp-subwhich activates the
virtualenv and then calls themainfunction in thepull_sp_submodule.Scripts created this way are slightly slower to start up than scripts
that directly run a Python file. Also, setuptools seems to do more
dependency checking when starting a script like this, so if you
regularly live with broken dependencies inside your virtualenv, this
will be frustrating for you. On the other hand, scripts made this way
will work better on Windows, if that's one of your target environments.Including compiled shared libraries in both source and binary packagesThis works because the NCBI Python/Linux environment is so homogeneous,
but it does cause problems - these compiled items are linux- and
architecture-specific, but this doesn't tell Python's packaging system
about that.  So for example if you runpip install applogon a Mac,
it will claim to succeed, but the library won't work.  See the next
section for how to do this in a more robust way.This includes things that use the C++ Toolkit (seepython-applogandcpp-toolkit-validatorsfor examples). These.sofiles should get
placed inside the python package hierarchy. Presumably, if you're
compiling them, they are build artifacts that won't be tracked by git,
so they won't be included automatically byauto-package-data. Instead,
once they are there, useextra_filesto have the packaging system
notice them:[files]
extra_files =
    ncbilog/libclog.so
    ncbilog/libclog.versionIf your packages live inside asrcdirectory, you do need to include
that in theextra_filespath:[files]
extra_files =
    src/mypkg/do_something_quickly.soNotice thatextra_filesis different fromdata_fileswhich we used
above.Including uncompiled C extensions (including Cython)Packit can coexist with setuptools's support for C extensions. Here is
anexample with a C file that will be compiled on the user's
system.
In that particular package, the author chose to require Cython for
developers but not for end users, so the distribution and the git repo
include both the.pyxfile and the.cfile it's translated to.Known IssuesIf your Python package is not in the root of your Git repository (sosetup.pyis not in the same directory as.git), thenauto-package-datawill not work.Theauto-package-datasection has configuration options, but they
don't do anything right now
(PY-504).Further DevelopmentAdd testsImprove docsMore configuration options for existing facilitiesNew facilitiesAllow extension through entry pointsUnlikesource env/bin/activate, this does not change the$PATHor set$VIRTUAL_ENV, so there are a few rare circumstances where
it's not good enough: if your script needs to start another script
usingsubprocessorpopen, or if it tries to access data using a
path relative to$VIRTUAL_ENV. Take a look atenv/bin/activate_this.pyif you encounter this problem.