Package: tags2sdists
tags2sdistsTags2sdists creates python sdists from tags into a structure that can serve as
a company-internal pypi (python package index).Basic operationTags2sdists looks at two directories:A source directory (“CHECKOUTDIR”) with checkouts. Every checkout (svn trunk
checkout, git/hg clone) is examined for tags according to that version
control system.A target directory (“SDISTDIR”) where per-package directories are made with
sdists named likePACKAGENAME-1.2.tar.gzin it.Those two directories are kept in sync by checking for packages/tags that are
available in the version control system but that are missing in the target
directory. If missing, an “sdist” (.tar.gz source distribution) is generated
and placed in the target directory.UsageTags2sdists provides thetags2sdistscommand:Usage: tags2sdists CHECKOUTDIR SDISTDIR
    CHECKOUTDIR: directory with checkouts
    SDISTDIR: directory with sdist package directories

Options:
  -a, --build-all Build all releases (=don't stop if the newest tag is found)
  -h, --help      Show this help message and exit
  -v, --verbose   Show debug output
  -q, --quiet     Show minimal outputWitn--build-all, all the tags are build. The default behaviour helps with
mis-behaving old tags, but if all the packages are clean,--build-allis a
good choice as also bugfix releases for older versions are build.SetupInstalling tags2sdists itself is as simple aspip install tags2sdistsoreasy_install tags2sdistsor including it in your buildout in the regular
manner.Next you need the CHECKOUTDIR and SDISTDIR directories.CHECKOUTDIR: you need a directory with checkouts. So doing it by hand is
fine. But when you use svn, a directory withsvn:externalsis probably
handiest. For everything else (and also for svn),checkoutmanageris the thing I’d use. Make a
config file (checkoutmanager.cfg) looking like this:[internalprojects]
vcs = git
basedir = /srv/packages/var/checkouts/
checkouts =
    git@github.com:lizardsystem/nensskel.git
    git@github.com:lizardsystem/lizard-ui.git
    git@github.com:lizardsystem/tags2sdists.gitAnd set up a cron job that runscheckoutmanager--configfile=YOURCONFIGFILE, it’ll update the checkouts in the base dir you
configured. (In that same cronjob, fire uptags2sdistsafterwards).SDISTDIR: just a directory somewhere will do. You’ll get a pypi-like
directory structure in there.A structure like generated with tags2sdists is a perfect index for
easy_install and buildout if you let apache host it.  Only problem: you can
only have one index (note: pip apparently supports multiple indexes).  You can
solve this problem by having apache redirect you to pypi when something is not
found.Here’s an example apache config snippet:# Allow indexing
Options +Indexes
IndexOptions FancyIndexing VersionSort

# Start of rewriterules to use our own var/private/* packages
# when available and to redirect to pypi if not.
RewriteEngine On
# Use our robots.txt:
RewriteRule ^/robots.txt - [L]
# Use our apache's icons:
RewriteRule ^/icons/.* - [L]
# We want OUR index.  Specified in a weird way as apache
# searches in a weird way for index.htm index.html index.php etc.
RewriteRule ^/index\..* - [L]

# Use our var/private/PROJECTNAME if available,
# redirect to pypi otherwise:
RewriteCond /path/on/server/var/private/$1 !-f
RewriteCond /path/on/server/var/private/$1 !-d
RewriteRule ^/([^/]+)/?$ http://pypi.python.org/pypi/$1/ [P,L]

# Use our var/private/PROJECTNAME/project-0.1.tar.gz if available,
# redirect to pypi otherwise:
RewriteCond /path/on/server/var/private/$1 !-d
RewriteRule ^/([^/]+)/([^/]+)$ http://pypi.python.org/pypi/$1/$2 [P,L]Using the apache-served indexYou can use such a custom apache-served index in two ways.  Easy_install has a-ioption for passing along an index:$> easy_install -i http://packages.my.server/ zest.releaserIn buildout, you can set it like this:[buildout]
index = http://packages.my.server/
parts =
    ...DevelopmentTo run the tests, installtoxglobally and just run it.For local testing, install it with virtualenv and pip:$ python3 -m venv .
$ bin/pip install -e .CreditsReinout van Reesstarted this package.Changelog of tags2sdists1.5 (2019-12-19)Small change in logging: make progress clearer by showing each package’s
name. This includes a list of missing sdists (if any).1.4.1 (2019-12-18)Fixed bug in condition from 1.4…1.4 (2019-12-18)Added –build-all option: don’t stop if the latest tag is found, but just
build all the tags.1.3 (2019-09-13)Updated setup (pytest, tox, travis-ci integration, etc).Made tags2sdists python3 compatible (tox tests 2.7 and 3.7).1.2 (2015-05-26)Compensating for newer setuptools versions.1.1 (2013-07-12)New way of searching for missing tags: we leave old
unneeded/wrong/renamed ones alone.Removing tags with ‘dev’ in their name from the list of missing
tags.1.0 (2013-01-15)Fixed the temp dir cleanup: under certain circumstances it left an
empty directory.0.7.1 (2011-12-08)Switching back to the correct directory after 0.7’s temp dir cleanup.0.7 (2011-12-08)Added more logging related to 0.6’s corner case.Cleaning up the temp dir after a run.0.6 (2011-12-07)Compensating for a corner case where a package was renamed.0.5 (2011-10-17)Fix for faulty setup.py’s in checkout directories. An error in there would
generate a directory namedTraceback\ (most\ recent\ call\ last):in the
sdist directory…0.4 (2011-10-12)Internally, the directories passed on the commandline are made
absolute. Necessary as there’s quite someos.chdir()going around.0.3 (2011-10-12)Added documentation.Renamed script from make_sdists to tags2sdists.0.2 (2011-10-11)Added script that combines the sdist tarball creation with the target
directory parsing and that generates all the necessary tarballs.Added creation (via zest.releaser) of a single sdist tarball.0.1 (2011-10-10)Added parsing of a target directory with sdists.Initial library skeleton created by nensskel.