minitage.recipe.scripts 1.58
zc.buildout recipes to compile and install software or python packages and generate scripts or configuration files sponsored by Makina Corpus.
License: BSD
Keywords: development buildout recipe

******************************************************************************
Recipe for compiling and installing software with or without minitage
******************************************************************************

.. contents::

=======================
Introduction
=======================

This is a collection of recipe which can be use inside or outside a minitage environment.
What is interresting in using them in minitage is that you ll have all your system dependencies in
the build environment automaticly.

The egg has those entry point:
    - *cmmi*: install configure/make/make install softwares
    - *fetch*: fetch something, somewhere, with git, http, frp, static, hg, svn or bzr.
    - *egg*: install python eggs / packages 'setuptoolisables'
    - *printer*: print or dump to a file all versions needed to achieve eggs
      requirements (versions.cfg made easy)
    - *scripts*: install scripts from an egg and install egg dependencies if they
      are not already in the cache
    - *wsgi*: Make a Python paste configuration file eatable by mod_wsgi with
      all the eggs dependencies you need.

The reasons why i have rewrite yet another buildout recipes builder are:
    - Support for downloading stuff
    - Do not rely on easy_install dependency system
    - Support on the fly patchs for eggs and other distribution.
    - Support multiple hooks at each stage of the build system.
    - Support for distutils
    - Robust offline mode
    - We like pypi, but offer a mode to scan for eggs without need to check
      the index,
    - Support malformed or not indexed distributions.
      In other terms, we provide an url, and the recipe builds it, that's all.
    - All recipes must support automaticly minitage dependencies and rpath linking.

You can browse the code on minitage's following resources:

    - http://git.minitage.org/git/minitage/eggs/minitage.recipe/
    - http://www.minitage.org/trac/browser/minitage/eggs/minitage.recipe

You can migrate your buldouts without any effort with buildout.minitagificator:

    * http://pypi.python.org/pypi/buildout.minitagificator

======================================
Makina Corpus sponsored software
======================================
|makinacom|_

* `Planet Makina Corpus <http://www.makina-corpus.org>`_
* `Contact us <mailto:python@makina-corpus.org>`_

  .. |makinacom| image:: http://depot.makina-corpus.org/public/logo.gif
  .. _makinacom:  http://www.makina-corpus.com





===============================================
minitage.recipe.scripts
===============================================


Abstract
-----------------
    - This recipe intends to install eggs and python software and on top of installed stuff, generating scripts and envrionment files.
    - This recipe inherit from minitage;recipe:egg.
    - Its heavilly inspired by zc.recipe.eggs* and try to completly replace it whereas be API compatbile.
    - You can use it in conjunction with the buildout.minitagificator extension which monkey patch zc.buildout to use minitage recipes.
    - What we can do that zc.recipe.egg wouldnt do, either at all or not in the way we want to:

        * All scripts support initialisation code
        * The 'scripts' egg metadata is also handled
    - You can use it as you would use minitage.recipe:egg, use patch facility and etc.
    - Ths recipe is also declared under those entry points:

        * minitage.recipe:eggs
        * minitage.recipe:script

Specific options
-----------------

    * All the shared options and the options from minitage.recipe:egg +
    * scripts

        - Scripts to generate, if empty, generate scripts for all the working set.
        - If your egg have an old 'scripts' metadata, and old scripts where you want wrappers to be generated, just add the egg name to the scripts entry.
        - If you want to rename a script, just enter something like entrypoint|scriptname=NewName::

            s=NewName

    * zap
      If you do not want to a script, just enter a line separated list of not wanted scripts

    * entry-points
        A list of entry-point identifiers of the form:::

            name=module:attrs

        where name is a script name, module is a dotted name resolving to a module name, and attrs is a dotted name resolving to a callable object within a module.
        This option is useful when working with distributions that don't declare entry points, such as distributions not written to work with setuptools.

    * interpreter
        The name of a script to generate that allow access to the Python interpreter with the PYTHONPATH set with all the working set entries.
    * dependent-scripts
      When set to true, scripts will be generated for all required eggs in addition to the eggs named specifically. This idea came from two forks of this recipe, repoze.recipe.egg and pylons_sandbox, but the option name is spelled with a dash instead of underscore and it defaults to false.
    * (script-name|interpretername-)arguments
        Specify some arguments to be passed to entry points as Python source.
    * (script-name|interpretername-)initialization
        Python code to run prior to call the entry point
    * env_initialization
	     run code after initialization of env (shell)
    * template-replacements

        * replacements for variables values in templates to override buildout's strip mecanism  (initialization, env_initialization, arguments)
        * Spaces before and after => are important:
        * Pattern are in the form::

			re-pattern => value




Detailled documentation
-------------------------

Generating all scripts
+++++++++++++++++++++++++++
Thus by not specifying any scripts entry in the buildout part.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir))     
    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... eggs=foo
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> noecho = [remove(os.path.join('eggs', egg)) for egg in os.listdir('eggs') if 'foo' in egg]
    >>> sh('bin/buildout -vvvvv install')
    b...
    minitage.recipe: Generated scripts: 's', 'toto.py'...

Look at what have been generated.
::

    >>> cat('bin', 'toto.py')
    #!...
    # ! GENERATED BY minitage.recipe !
    import os
    import sys
    import subprocess...
    sys.path[0:0] = ['/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
    # EXEC ORGINAL CODE WITHOUT SHEBANG
    __doc__  = 'I am generated by minitage.recipe.script recipe'...
    os.environ['PYTHONPATH'] = ':'.join(sys.path + os.environ.get('PYTHONPATH', '').split(':'))
    sys.argv.pop(0)
    sys.exit(
        subprocess.Popen(
            [sys.executable, '/tmp/buildout.test/eggs/foo-1.0-py....egg/EGG-INFO/scripts/toto.py']+sys.argv,
            env=os.environ
        ).wait()
    )...
    >>> cat('bin', 's')
    #!...
    #!!! #GENERATED VIA MINITAGE.recipe !!!...
    import sys...
    sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
    import toto.toto...
    if __name__ == '__main__':
        toto.toto.f()...



Selecting scripts to install
+++++++++++++++++++++++++++++++
Installing only s.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir))    
    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... scripts =
    ...     s
    ... eggs=foo
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...
    minitage.recipe: Generated scripts: 's'...

Installing only toto.py.
::

    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... scripts =
    ...     toto.py
    ... eggs=foo
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...
    minitage.recipe: Generated scripts: 'toto.py'....

.. desactivated because caused more harm than good... too much scripts no filtered
.. Installing scripts from the foo distribution.
..
..     >>> data = """
..     ... [buildout]
..     ... download-cache=${buildout:directory}/dl
..     ... parts = part
..     ... [part]
..     ... recipe=minitage.recipe.scripts
..     ... find-links=%(index)s
..     ... scripts =
..     ...     foo
..     ... eggs=foo
..     ... """%{'index': index_url}
..     >>> touch('buildout.cfg', data=data)
..     >>> sh('bin/buildout -vvvvv install')
..     b...
..     minitage.recipe: Generated scripts: 's', 'toto.py'....


Declaring entry-points
+++++++++++++++++++++++
We ll add an entry point 't' to be generated.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir))    
    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... entry-points=t=toto.toto:f
    ... eggs=foo
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...
    minitage.recipe: Generated scripts: 't'...
    >>> cat('bin', 't') #doctest: +REPORT_NDIFF
    #!...
    #!!! #GENERATED VIA MINITAGE.recipe !!!
    <BLANKLINE>
    import sys
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-...
    ...]
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    import toto.toto
    <BLANKLINE>
    if __name__ == '__main__':
        toto.toto.f()
    <BLANKLINE>


Adding initialization code
++++++++++++++++++++++++++++
What about adding environment variables for gis env.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir))   
    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... entry-points=t=toto.toto:f
    ... eggs=foo
    ... initialization = import os;os.environ.set('GDAL', 'TRUE')
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...

::

    >>> "import os;os.environ.set('GDAL', 'TRUE')" in open(os.path.join('bin', 't')).read()
    True



Adding arguments
++++++++++++++++++
What about adding arguments to our launchers.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir))   
    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... eggs = foo
    ... entry-points=t=toto.toto:f
    ... arguments = ['a', 'b']
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...

::

    >>> "toto.toto.f(['a', 'b'])" in open(os.path.join('bin', 't')).read()
    True


Generating a python interpreter
++++++++++++++++++++++++++++++++++
Here is how you can generate a specific python interpreter will all the environement of the working set.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir))  
    >>> data = """
    ... [buildout]
    ... eggs-directory =${buildout:directory}/eggs
    ... download-cache=${buildout:directory}/dl
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... interpreter = mypy
    ... arguments = ['a', 'b']
    ... eggs=foo
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...
    minitage.recipe: Generated scripts: 'mypy',...

::

    >>> cat('bin', 'mypy')
    #!...
    #!!! #GENERATED VIA MINITAGE.recipe !!!...
    sys.path[0:0] = [ '/tmp/buildout.test/eggs/foo-1.0-py....egg' ]...
    if _interactive:
        import code
        code.interact(banner="", local=globals())...



Generating an envrionment file
++++++++++++++++++++++++++++++++++
Here is how you can generate a specific envrionment file that you can source from to get the PYTHONPATH populated with eggs that you have configured.
::

    >>> init_env()
    >>> index_url = start_server(os.path.sep.join(tempdir)) 
    >>> data = """
    ... [buildout]
    ... download-cache=${buildout:directory}/dl
    ... eggs-directory =${buildout:directory}/eggs
    ... parts = part
    ... [part]
    ... recipe=minitage.recipe.scripts
    ... find-links=%(index)s
    ... env-file = mypy
    ... eggs=foo
    ... """%{'index': index_url}
    >>> touch('buildout.cfg', data=data)
    >>> sh('bin/buildout -vvvvv install')
    b...
    >>> cat('bin', 'mypy')
    #!/usr/bin/env sh
    <BLANKLINE>
    PYTHONPATH="/tmp/buildout.test/eggs/foo-1.0..."
    export PYTHONPATH
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>


=======================
CHANGELOG
=======================

1.58 (2013-03-31)
-----------------

- fix bug that would save scripts from eggs with extras beeing generated in some corner case


1.57 (2013-02-13)
-----------------

- buildout2 compat


1.56 (2013-02-13)
-----------------
- Keep return code from scripts


1.55 (2011-03-11)
-----------------

    - Use os.execvpe to spawn processes, not subprocess. Thx to Dave Foster (daf)
    - Split tests
    - Add initialization for interpreter script
    - Fix type 'list'and pattern replacement code

1.50
------

    - workaround to install when there are conflicts from versions pinned in setup.py and different ones in buildout


1.47
------

    - Added the dependent-scripts option to stick to zc.recipe.egg

1.45
----

	- better handle of entrypoints/console script conflict options (cs. prefix, see code)
	- add per script options (initialization, argurments, python)

1.44
-----

	- when both EntryPoint and Script have the same name, it's the EP which wins.


1.41
-----

    - fix a special case where all scripts were filtered out

1.39
--------

    -   do not preerve PYTHONPATH on env.file generation

1.37
-------

    - Decorator helper


1.36
-----

    - fix develop link

1.35
---------

    - splitted out from minitage.recipe