Package: hexagonit.recipe.cmmi
ContentsChange History2.0 (2013-04-07)1.6 (2012-06-28)1.5.1 (2012-05-21)1.5.0 (2010-12-17)1.4.0 (2010-08-27)1.3.1 (2010-08-23)1.3.0 (2009-09-20)1.2.01.1.11.1.01.0.11.0.0Detailed DocumentationSupported optionsExample usageInstalling a Perl packageInstalling a package without anautoconflike systemInstalling checkoutsAdvanced configurationCustomizing the build processContributorsDownloadThe recipe provides the means to compile and install source distributions
usingconfigureandmakeand other similar tools. It is inspired by
thezc.recipe.cmmirecipe but provides more control over the build process.Repository:http://github.com/hexagonit/hexagonit.recipe.cmmiClone URL: git clone git://github.com/hexagonit/hexagonit.recipe.cmmi.gitIssue tracker:http://github.com/hexagonit/hexagonit.recipe.cmmi/issuesSupported Python versions: 2.6, 2.7, 3.2, 3.3Supported zc.buildout versions: 1.x, 2.xTravis build:Change History2.0 (2013-04-07)Dropped support for Python 2.4/2.5. Use the 1.x versions for those.
Currently supported versions are Python 2.6, 2.7, 3.2 and 3.3. [dokai]Python 3.2/3.3 support. [dokai, mmariani]Tox support.https://github.com/hexagonit/hexagonit.recipe.cmmi/pull/7[msabramo]Travis CI integration.https://travis-ci.org/hexagonit/hexagonit.recipe.cmmiSeehttps://github.com/hexagonit/hexagonit.recipe.cmmi/pull/7[msabramo]Automatically clean up a left-over compile directory from a previously
failed run. Seehttps://github.com/hexagonit/hexagonit.recipe.cmmi/pull/9[desaintmartin]1.6 (2012-06-28)Relicensed under the3-clause BSD license.
[dokai]1.5.1 (2012-05-21)PEP8 / Pyflakes cleanup.
[dokai]Quote--prefixpaths to support whitespace characters in
the path. Closeshttps://github.com/hexagonit/hexagonit.recipe.cmmi/pull/4[galpin]Fixed importing of the hook scripts on Windows. Closeshttps://github.com/hexagonit/hexagonit.recipe.cmmi/pull/6[grzn]1.5.0 (2010-12-17)Refactored the environment variable handling logic. Python versions prior
to 2.6 have an issue clearing the environment variables usingos.environ.clear()(Seehttp://bugs.python.org/issue3227). [dokai]Instead of modifyingos.environdirectly we use thesubprocessmodule to run the commands in child processes which are given an explicit
environment which is a copy of the currentos.environaugmented with
the per-part overrides. As a result,os.environis no longer modified
by this recipe.ThePython hook scriptsare passed the augmented environment dictionary
as a third parameter.WarningExisting hook scripts accepting only two parameters
continue to work but they do not have access to the modified
environment variables. To fix this they should be refactored
to accept the third parameter.Seehttps://github.com/hexagonit/hexagonit.recipe.cmmi/issues/issue/1/#issue/1/comment/605362for details.1.4.0 (2010-08-27)Added support for passing options tomakewith the newmake-optionsoption. See theInstalling a package without an autoconf
like systemsection below for an example. [dokai]The--prefixparameter will be automatically given to the configure
command if and only iftheconfigure-commandis not used to specify a custom configure command and--prefixis not given explicitly in theconfigure-optionsoption.[dokai]Removed theis_build_dir()heuristic.Previously the recipe inspected the contents of the downloaded package to
determine if it contained the necessary files for building the package (it
checked if files namedconfigureorMakefile.PLexisted) and gave
an error message if they were missing. However, the recipe is useful for
building many different kinds of software packages and checking for
particular files limited its use severely.Now the recipe omits any checks for particular files in the downloaded
package. It is recommended that you use themd5sumoption in your part
configuration to assert that you are downloading the package you expect
to. [dokai]1.3.1 (2010-08-23)Refactored theis_build_dir()helper method to make it easier to test
and override in customized recipes. [dokai]Fixed the handling of the working directory so that it is restored to the
state before executing the recipe regardless whether the recipe was
successful or not. Thanks to Jonathan Ballet for the report and preliminary
patch. [dokai]Fixedhttp://github.com/hexagonit/hexagonit.recipe.cmmi/issues#issue/1Environment variables defined in one part will no longer leak to other
subsequent parts. [dokai]1.3.0 (2009-09-20)Added new optionsenvironment-sectionandenvironmentto control
environment variables before executing the recipe.Added a new option,prefix, to override the installation prefix.
Defaults to the previously hardcoded value of the part location.1.2.0Added newconfigure-commandoption to control the command used to
generate theMakefile. This makes it possible to build slightly
different packages, e.g. Perl projects where Makefile.PL replaces the
configure script.1.1.1Don’t try to execute hooks if the option is an empty string. This
will make it possible to disable hooks when extending existing
parts.1.1.0Added new optionpathto allow building and installing local
source trees. Thepathoption is mutually exclusive withurl.1.0.1Fixed a bug with ‘keep-compile-dir’ option. The location of the
compilation directory was not available through theoptions['compile-directory']option as documented.1.0.0Initial public release.Detailed DocumentationSupported optionsurlURL to the package that will be downloaded and extracted. The
supported package formats are .tar.gz, .tar.bz2, and .zip. The
value must be a full URL,
e.g.http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz. Thepathoption can not be used at the same time withurl.pathPath to a local directory containing the source code to be built
and installed. The directory must contain theconfigurescript. Theurloption can not be used at the same time withpath.prefixCustom installation prefix passed to the--prefixoption of theconfigurescript. Defaults to the location of the part. Note that this
is a convenience shortcut which assumes that the defaultconfigurecommand is used to configure the package. If theconfigure-commandoption is used to define a custom configure command no automatic--prefixinjection takes place. You can also set the--prefixparameter explicitly inconfigure-options.md5sumMD5 checksum for the package file. If available the MD5
checksum of the downloaded package will be compared to this value
and if the values do not match the execution of the recipe will
fail.make-binaryPath to themakeprogram. Defaults to ‘make’ which
should work on any system that has themakeprogram available
in the systemPATH.make-optionsExtraKEY=VALUEoptions included in the invocation of themakeprogram. Multiple options can be given on separate lines to increase
readability.make-targetsTargets for themakecommand. Defaults to ‘install’
which will be enough to install most software packages. You only
need to use this if you want to build alternate targets. Each
target must be given on a separate line.configure-commandName of the configure command that will be run to generate the Makefile.
This defaults to./configurewhich is fine for packages that come with
a configure script. You may wish to change this when compiling packages
with a different set up. See theCompiling a Perl packagesection for
an example.configure-optionsExtra options to be given to theconfigurescript. By default
only the--prefixoption is passed which is set to the part
directory. Each option must be given on a separate line.patch-binaryPath to thepatchprogram. Defaults to ‘patch’ which should
work on any system that has thepatchprogram available in the
systemPATH.patch-optionsOptions passed to thepatchprogram. Defaults to-p0.patchesList of patch files to the applied to the extracted source. Each
file should be given on a separate line.pre-configure-hookCustom python script that will be executed before running theconfigurescript. The format of the options is:/path/to/the/module.py:name_of_callablewhere the first part is a filesystem path to the python module and the
second part is the name of the callable in the module that will be called.
The callable will be passed three parameters in the following order:Theoptionsdictionary from the recipe.The globalbuildoutdictionary.A dictionary containing the currentos.environaugmented with
the part specific overrides.The callable is not expected to return anything.NoteTheos.environis not modified so if the hook script is
interested in the environment variable overrides defined for the
part it needs to read them from the dictionary that is passed in
as the third parameter instead of accessingos.environdirectly.pre-make-hookCustom python script that will be executed before runningmake. The format and semantics are the same as with thepre-configure-hookoption.post-make-hookCustom python script that will be executed after runningmake. The format and semantics are the same as with thepre-configure-hookoption.keep-compile-dirSwitch to optionally keep the temporary directory where the
package was compiled. This is mostly useful for other recipes that
use this recipe to compile a software but wish to do some
additional steps not handled by this recipe. The location of the
compile directory is stored inoptions['compile-directory'].
Accepted values aretrueorfalse, defaults tofalse.environment-sectionName of a section that provides environment variables that will be used to
augment the variables read fromos.environbefore executing the
recipe.This recipe does not modifyos.environdirectly. External commands
run as part of the recipe (e.g. make, configure, etc.) get an augmented
environment when they are forked. Python hook scripts are passed the
augmented as a parameter.The values of the environment variables may contain references to other
existing environment variables (including themselves) in the form of
Python string interpolation variables using the dictionary notation. These
references will be expanded using values fromos.environ. This can be
used, for example, to append to thePATHvariable, e.g.:[component]
recipe = hexagonit.recipe.cmmi
environment-section =
    environment

[environment]
PATH = %(PATH)s:${buildout:directory}/binenvironmentA sequence ofKEY=VALUEpairs separated by newlines that define
additional environment variables used to updateos.environbefore
executing the recipe.The semantics of this option are the same asenvironment-section. If
bothenvironment-sectionandenvironmentare provided the values from
the former will be overridden by the latter allowing per-part customization.Additionally, the recipe honors thedownload-cacheoption set
in the[buildout]section and stores the downloaded files under
it. If the value is not set a directory calleddownloadswill be
created in the root of the buildout and thedownload-cacheoption set accordingly.The recipe will first check if there is a local copy of the package
before downloading it from the net. Files can be shared among
different buildouts by setting thedownload-cacheto the same
location.Example usageWe’ll use a simple tarball to demonstrate the recipe.>>> import os.path
>>> src = join(os.path.dirname(__file__), 'testdata')
>>> ls(src)
- Foo-Bar-0.0.0.tar.gz
- haproxy-1.4.8-dummy.tar.gz
- package-0.0.0.tar.gzThe package contains a dummyconfigurescript that will simply
echo the options it was called with and create aMakefilethat
will do the same.Let’s create a buildout to build and install the package.>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = hexagonit.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... """ % src)This will download, extract and build our demo package with the
default build options.>>> print(system(buildout))
Installing package.
package: Extracting package to /sample_buildout/parts/package__compile__
configure --prefix=/sample_buildout/parts/package
building package
installing packageAs we can see the configure script was called with the--prefixoption by default followed by calls tomakeandmake install.Installing a Perl packageThe recipe can be used to install packages that use a slightly different build
process. Perl packages often come with aMakefile.PLscript that performs
the same task as aconfigurescript and generates aMakefile.We can build and install such a package by overriding theconfigure-commandoption. The following example builds a Foo::Bar perl module and installs it in
a custom location within the buildout:>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = foobar
... perl_lib = ${buildout:directory}/perl_lib
...
... [foobar]
... recipe = hexagonit.recipe.cmmi
... configure-command = perl -I${buildout:perl_lib}/lib/perl5 Makefile.PL INSTALL_BASE=${buildout:perl_lib}
... url = file://%s/Foo-Bar-0.0.0.tar.gz
... """ % src)

>>> print(system(buildout))
Uninstalling package.
Installing foobar.
foobar: Extracting package to /sample_buildout/parts/foobar__compile__
building package
installing packageInstalling a package without anautoconflike systemSome packages do not use a configuration mechanism and simply provide aMakefilefor building. It is common in these cases that the build process
is controlled entirely by direct options tomake. We can build such a
package by faking a configure command that does nothing and passing the
appropriate options tomake. Thetrueutility found in most shell
environments is a good candidate for this although anything that returns a
zero exit code would do.We are using a dummy “HAProxy” package as an example of a package with only a
Makefile and using explicitmakeoptions to control the build process.>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = haproxy
...
... [haproxy]
... recipe = hexagonit.recipe.cmmi
... configure-command = true
... make-options =
...     TARGET=linux26
...     CPU=i686
...     USE_PCRE=1
... url = file://%s/haproxy-1.4.8-dummy.tar.gz
... """ % src)>>> print(system(buildout))
Uninstalling foobar.
Installing haproxy.
haproxy: Extracting package to /sample_buildout/parts/haproxy__compile__
Building HAProxy 1.4.8 (dummy package)
TARGET: linux26
CPU: i686
USE_PCRE: 1
Installing haproxyInstalling checkoutsSometimes instead of downloading and building an existing tarball we need to
work with code that is already available on the filesystem, for example an SVN
checkout.Instead of providing theurloption we will provide apathoption to
the directory containing the source code.Let’s demonstrate this by first unpacking our test package to the filesystem
and building that.>>> checkout_dir = tmpdir('checkout')
>>> import setuptools.archive_util
>>> setuptools.archive_util.unpack_archive('%s/package-0.0.0.tar.gz' % src,
...                                        checkout_dir)
>>> ls(checkout_dir)
d package-0.0.0>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = hexagonit.recipe.cmmi
... path = %s/package-0.0.0
... """ % checkout_dir)>>> print(system(buildout))
Uninstalling haproxy.
Installing package.
package: Using local source directory: /checkout/package-0.0.0
configure --prefix=/sample_buildout/parts/package
building package
installing packageSince using thepathimplies that the source code has been acquired
outside of the control of the recipe also the responsibility of managing it is
outside of the recipe.Depending on the software you may need to manually runmake cleanetc.
between buildout runs if you make changes to the code. Also, thekeep-compile-dirhas no effect whenpathis used.Advanced configurationThe above options are enough to build most packages. However, in some cases it
is not enough and we need to control the build process more. Let’s try again
with a new buildout and provide more options.>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [build-environment]
... CFLAGS = -I/sw/include
... LDFLAGS = -I/sw/lib
...
... [package]
... recipe = hexagonit.recipe.cmmi
... url = file://%(src)s/package-0.0.0.tar.gz
... md5sum = 6b94295c042a91ea3203857326bc9209
... prefix = /somewhere/else
... environment-section = build-environment
... environment =
...     LDFLAGS=-L/sw/lib -L/some/extra/lib
... configure-options =
...     --with-threads
...     --without-foobar
... make-targets =
...     install
...     install-lib
... patches =
...     patches/configure.patch
...     patches/Makefile.dist.patch
... """ % dict(src=src))This configuration uses custom configure options, an environment section,
per-part customization to the environment, custom prefix, multiple make
targets and also patches the source code before the scripts are run.>>> print(system(buildout))
Uninstalling package.
Installing package.
package: [ENV] CFLAGS = -I/sw/include
package: [ENV] LDFLAGS = -L/sw/lib -L/some/extra/lib
package: Extracting package to /sample_buildout/parts/package__compile__
package: Applying patches
patching file configure
patching file Makefile.dist
patched-configure --prefix=/somewhere/else --with-threads --without-foobar
building patched package
installing patched package
installing patched package-libCustomizing the build processSometimes even the above is not enough and you need to be able to control the
process in even more detail. One such use case would be to perform dynamic
substitutions on the source code (possible based on information from the
buildout) which cannot be done with static patches or to simply run arbitrary
commands.The recipe allows you to write custom python scripts that hook into the build
process. You can define a script to be run:before the configure script is executed (pre-configure-hook)before the make process is executed (pre-make-hook)after the make process is finished (post-make-hook)Each option needs to contain the following information/full/path/to/the/python/module.py:name_of_callablewhere the callable object (here name_of_callable) is expected to take three
parameters:Theoptionsdictionary from the recipe.The globalbuildoutdictionary.A dictionary containing the currentos.environaugmented with
the part specific overrides.These parameters should provide the callable all the necessary information to
perform any part specific customization to the build process.Let’s create a simple python script to demonstrate the functionality. You can
naturally have separate modules for each hook or simply use just one or two
hooks. Here we use just a single module.>>> hooks = tmpdir('hooks')
>>> write(hooks, 'customhandlers.py',
... """
... import logging
... log = logging.getLogger('hook')
...
... def preconfigure(options, buildout, environment):
...     log.info('This is pre-configure-hook!')
...
... def premake(options, buildout, environment):
...     log.info('This is pre-make-hook!')
...
... def postmake(options, buildout, environment):
...     log.info('This is post-make-hook!')
...
... """)and a new buildout to try it out>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = hexagonit.recipe.cmmi
... url = file://%(src)s/package-0.0.0.tar.gz
... pre-configure-hook = %(module)s:preconfigure
... pre-make-hook = %(module)s:premake
... post-make-hook = %(module)s:postmake
... """ % dict(src=src, module='%s/customhandlers.py' % hooks))>>> print(system(buildout))
Uninstalling package.
Installing package.
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure-hook
hook: This is pre-configure-hook!
configure --prefix=/sample_buildout/parts/package
package: Executing pre-make-hook
hook: This is pre-make-hook!
building package
installing package
package: Executing post-make-hook
hook: This is post-make-hook!For even more specific needs you can write your own recipe that useshexagonit.recipe.cmmiand set thekeep-compile-diroption totrue.
You can then continue from where this recipe finished by reading the location
of the compile directory fromoptions['compile-directory']from your own
recipe.ContributorsKai Lautaportti (dokai), AuthorCédric de Saint Martin (desaintmartin)Marc Abramowitz (msabramo)Nicolas Dumazet (nicdumz)Guy Rozendorn (grzn)Marco Mariani (mmariani)galpinDownload