Package: buildout.eggtractor
buildout.eggtractorQ: What is a buildout extension ?A:http://pypi.python.org/pypi/zc.buildout#extensionsThe problemWhen developing zope/plone eggs with buildout I have to edit the buildout
configuration file ( in 3 places ) each time I create/delete/rename a
development egg in thesrcdirectory or in other development directories
(sometime I have more than one).I have to add/delete/rename the egg in theeggsoption of the[buildout]and then add/delete/rename the egg path in thedevelopoption
of the[buildout]and in the end add/delete/rename thezcmloption of
the zope[instance]or in theconfigure.zcmlfile of my policy package.
This is too much specially when the speed is set to development mode. I need a
less boring way to develop.Solutionbuildout.eggtractoris a buildout extension that scans thesrcdirectory or a list of directories I give for eggs and picks them up
automatically. So no more editing of the buildout’s configuration file.Whenbuildout.eggtractorfinds an egg in the scanned directory it:1. adds the egg to the ``eggs`` option of all zope instance parts or
   to a set of given parts

2. adds the egg's path in the ``develop`` option of the ``[buildout]``

3. If ``tractor-autoload-zcml`` is not given or set to other thing than
   false, scans the egg folder for ``configure.zcml``, ``meta.zcml`` and
   ``overrides.zcml`` and adds the appropriate zcml entries to the ``zcml``
   option of the zope instance parts or to a set of given parts.This steps are done on the fly when running buildout. So I can add/delete/rename
an egg and it will be picked up.NOTE: The extension does not write to the buildout’s configuration file.buildout.eggtractor optionstractor-src-directory:A set of directories to scan for development eggs. Defaults to thesrcdirectory of the buildout.tractor-target-parts:A set of parts to update theireggsoption with eggs found in thetractor-src-directory. Defaults to zope instance parts if any.tractor-autoload-zcml(boolean):Update thezcmloption oftractor-target-partswith the eggs found
intractor-src-directory. Defaults to truetractor-zcml-top:A set of eggs to load their zcml files first. Defaults to an empty set.How to use itUsingbuildout.eggtractoris very simple. As said, it is a buildout
extension. All I have to do is to declare it in theextensionsoption:[buildout]
parts =

extensions = buildout.eggtractorThat’s all.buildout.eggtractorwill scan thesrcdirectory and do its
job every time I run the buildout command.When I have other directories I want to scan I just add antractor-src-directoryoption in the[buildout]and add my directories
there:[buildout]
parts =

extensions = buildout.eggtractor

tractor-src-directory =
                      dev-src1
                      dev-src2
                      srcIn a few cases when the priority of loading zcml files matters. I add the egg to
be loaded first in thetractor-zcml-topoption in the[buildout]:[buildout]
parts =

extensions = buildout.eggtractor

tractor-src-directory =
          dev-src1
          dev-src2
          src

tractor-zcml-top =
          plone.app.mypackage1If I want to add the eggs found in the development directories to the eggs
option of a given set of parts, I add atractor-target-partsoption
and add the parts there:[buildout]
parts = instance1 instance2 instance3

extensions = buildout.eggtractor

tractor-target-parts = instance1 instance3This way only instance1 and instance3 will be updated.If I have already other way to include the zcml files (ie: z3c.autoinclude)
and don’t wanteggtractorto generate the zcml slugs, I add antractor-autoload-zcmloption and set it tofalseIn most cases you will only need to addbuildout.eggtractorto theextensionsoption of the[buildout]without any extra configuration
options.LIMITATIONThe extension assumes that the egg name reflects its file system structureexample: if the egg name is com.mustap.www the extension assumes that the file
system structure is one of the following:1. com.mustap.www/src/com/mustap/www

2. com.mustap.www/com/mustap/wwwThis is where the extension looks for configure.zcml, meta.zcml and
overrides.zcml files.If the egg name has nothing to do with how it is structured on the system,
the extension will ignore it.XXX: I guess walking through the directory is better than this assumption.In my case this is not a limitation as I choose my egg names that way.Mustaphaemail: mustap_at_gmail_comweb:http://www.mustap.comChange history0.6 (2008-10-29)fixed install problem on windows:http://plone.org/support/forums#nabble-f293351split buildout configuration values before we test if an egg was
added already. using ‘in’ with a string doen’t work if an egg’s
name is a substring of an already added egg (e.g. plone.app.content
after plone.app.contentmenu).
[csenger]sort output of os.listdir (it’s order is undefined) to make
debugging easier.
[csenger]0.5 (2008-04-30)Refactoring
Added tractor-target-parts option
Added tractor-autoload-zcml option
Updated documentation
Updated tests: need more tests
[mustapha]0.4 (2008-04-27)Made sure the configure.zcml is added to package includes if a meta.zcml or
overrides.zcml in the same packages have already been found.
[hannosch]0.3 (2008-04-27)Use a new line as a separator for added entries. A space makes buildout
think it has to deal with a version specifier.
[hannosch]0.2 (2008-04-27)Added support for automatically finding multiple instances.
[hannosch]Better use the recipe name for finding the instance, as this is less likely
to conflict.
[hannosch]0.1 (2008-04-27)Whitespace fixes.
[hannosch]Created recipe with ZopeSkel.
[mustapha]Detailed DocumentationTests for buildout.eggtractor buildout extensionLet’s create a buildout configuration file:>>> data = """
... [buildout]
... parts = zope2 instance1 instance2
... extensions = buildout.eggtractor
... eggs =
... develop =
... [instance1]
... recipe = plone.recipe.zope2instance
... zope2-location = ${zope2:location}
... user = admin:admin
... [instance2]
... recipe = plone.recipe.zope2instance
... zope2-location = ${zope2:location}
... user = admin:admin
... [zope2]
... recipe = plone.recipe.zope2install
... url = http://www.zope.org/Products/Zope/2.9.8/Zope-2.9.8-final.tgz
... """
>>> rmdir(tempdir, 'buildout.test')
>>> cd(tempdir)
>>> sh('mkdir buildout.test')
mkdir buildout.test
<BLANKLINE>
>>> cd('buildout.test')
>>> touch('buildout.cfg', data=data)
>>> ls('.')
buildout.cfgrun the buildout first time so wget our zope instances:>>> sh('buildout bootstrap')
buildout bootstrap
Creating directory '/tmp/buildout.test/bin'.
Creating directory '/tmp/buildout.test/parts'.
Creating directory '/tmp/buildout.test/develop-eggs'.
Generated script '/tmp/buildout.test/bin/buildout'.
<BLANKLINE>
>>> sh('bin/buildout')
bin/buildout
...
Installing instance1.
Generated script '/tmp/buildout.test/bin/instance1'.
Generated script '/tmp/buildout.test/bin/repozo'.
Installing instance2.
Generated script '/tmp/buildout.test/bin/instance2'.
<BLANKLINE>
<BLANKLINE>Now let’s create an egg in the src directory:>>> sh("paster create --no-interactive -o src -t plone_app com.mustap.www namespace_package=com namespace_package2=mustap package=www")
paster create --no-interactive -o src -t plone_app com.mustap.www namespace_package=com namespace_package2=mustap package=www
...
...setup.py egg_info
<BLANKLINE>Ok, so now that we have an egg, lets run the buildout in offline mode. We
should get a link file in develop-egg, a zcml slugs in
parts/instance1/etc/package-includes and parts/instance2/etc/package-includes
and a line with the path to our egg in the bin/instance1 and bin/instance2
files.First we check that there is nothing of the previous mentioned things:>>> ls('develop-eggs')
>>> ls('parts/instance1/etc/package-includes')
No directory named parts/instance1/etc/package-includes

>>> ls('parts/instance2/etc/package-includes')
No directory named parts/instance2/etc/package-includes

>>> sh('grep com.mustap.www bin/instance1')
grep com.mustap.www bin/instance1
<BLANKLINE>

>>> sh('grep com.mustap.www bin/instance2')
grep com.mustap.www bin/instance2
<BLANKLINE>OK, now run the buildout in offline mode:>>> sh('./bin/buildout -o')
./bin/buildout -o
...Check that we have a correct created buildout. First check that we have
a link in the develop-eggs directory:>>> ls('develop-eggs')
com.mustap.www.egg-linkCheck that we have our zcml slugs in the package-includes:>>> ls('parts', 'instance1', 'etc', 'package-includes')
001-com.mustap.www-configure.zcml

>>> ls('parts', 'instance2', 'etc', 'package-includes')
001-com.mustap.www-configure.zcmland in the end check that there is a line in bin/instance1 and bin/instance1
that includes our egg in the path:>>> cat('bin', 'instance1')
#!/usr/bin/python2.4
...
sys.path[0:0] = [
  '/tmp/buildout.test/src/com.mustap.www',
...
  ]
...

>>> cat('bin', 'instance2')
#!/usr/bin/python2.4
...
sys.path[0:0] = [
  '/tmp/buildout.test/src/com.mustap.www',
...
  ]
...Let’s now try thetractor-target-partsoption. We create a new buildout.cfg file
with an emptytractor-target-parts:>>> data = data.replace('eggs =', 'tractor-target-parts = \neggs = ')
>>> touch('buildout.cfg', data=data)
>>> sh('./bin/buildout -o')
./bin/buildout -o
...We get the egg link in the develop-egg directory:>>> ls('develop-eggs')
com.mustap.www.egg-linkBut no zcml slug in the instance 1 and 2:>>> ls('parts', 'instance1', 'etc', 'package-includes')
No directory named parts/instance1/etc/package-includes

>>> ls('parts', 'instance2', 'etc', 'package-includes')
No directory named parts/instance2/etc/package-includesNor a line in bin/instance1 and bin/instance2 with our egg path:>>> code = cat('bin', 'instance1', returndata=True)
>>> code.find('com.mustap.www') == -1
True

>>> code = cat('bin', 'instance2', returndata=True)
>>> code.find('com.mustap.www') == -1
TrueBut if thetractor-target-partsoption is not empty:>>> data = data.replace('tractor-target-parts =', 'tractor-target-parts = instance1')
>>> touch('buildout.cfg', data=data)
>>> sh('./bin/buildout -o')
./bin/buildout -o
...and we get a zcml slug only in the specified target:>>> ls('parts', 'instance1', 'etc', 'package-includes')
001-com.mustap.www-configure.zcml

>>> ls('parts', 'instance2', 'etc', 'package-includes')
No directory named parts/instance2/etc/package-includesand only the specified target’s control script is updated:>>> code = cat('bin', 'instance1', returndata=True)
>>> code.find('com.mustap.www') == -1
False

>>> code = cat('bin', 'instance2', returndata=True)
>>> code.find('com.mustap.www') == -1
TrueContributorsmustapha, Authorhannosch, Minor fixes