Package: rod.recipe.appengine
The rod.recipe.appengine is a zc.buildout recipe to build, test and deploy
projects for the Google App Engine. It makes it easy to use eggs and resolve
their dependencies automatically.To be honest, this is a recipe for scrambled eggs. It produces a zip file
containing all configured external packages in a traditional folder hierarchy.Copyright and licenseCopyright 2009, 2010, 2011 Tobias RodaebelThis software is released under the GNU Lesser General Public License,
Version 3.CreditsThanks to Attila Olah who provided a patch which fixes two issues where the
recipe wasn’t able to find the original pkg_resources.py file. He also enabled
the recipe to be used together with distribute as a replacement for setuptools.Attila Olah also provided a patch for supporting regular expressions when using
the exclude option.Thanks to Tom Lynn for submitting a patch which fixes an issue with symlink on
platforms other than UNIX.Lots of thanks to Gael Pasgrimaud for submitting a patch that allows for
zc.recipe.egg options such as extra-paths and entry-points.A brief documentationThis recipe takes a number of options:appengine-libPath to an already installed appengine libraryappserver-script-nameThe name of the script in GAE to run the dev. server.
The default is ‘dev_appserver.py’. Because there have been
intermediate versions of the GAE SDK using ‘dev_appserver2.py’,
this script name is configurable.appserver-run-fileThe name of the run function in the “appserver-script-name” script.
This name changed in GAE SDK > 1.7.5. Default is “_run_file”.
In previous GAE versions it was “run_file”. This switch enables
the use of this recipe for older GAE versions.eggsList of required eggsentry-pointsA list of entry-point identifiers. See zc.recipe.egg for a more detailed
documentation.excludeA list of basenames and regular expressions to be excluded when setting up
the application files, e.g. the whole ‘tests’ directory.ignore-packagesList of packages that need to be ignored when setting up the applicaton files.extra-pathsExtra paths to include in a generated script.initializationSpecify some Python initialization code. This is very limited. In
particular, be aware that leading whitespace is stripped from the code
given.packagesA list of packages to be included into the zip archive, which will be
uploaded to the appspot.patchSpecifies a patch file for patching the SDK source-tree. This option is
not allowed if appengine-lib is specified.patch-optionsList of patch options separated by blanks. (Default=-p1)server-scriptThe name of the script to run the development server.srcThe directory which contains the project source files.symlink-gae-runtimeWhen this flag is True, a symlink to the “_python_runtime.py”
script is created in the buildout ‘bin’ directory.
This script is needed for newer versions of the GAE SDK.
You have to manually remove the symlink when you change this
flag from True to False.urlThe url for fetching the google appengine distributionuse_setuptools_pkg_resourcesFlag whether the recipe shall copy setuptool’s pkg_resources.py into the
app directory rather than writing a dummy version. (Default=False)zip-nameThe name of the zip archive containing all external packages ready
to deploy.zip-packagesFlag whether external packages shall be zipped into a single zip file.
(Default=True)TestsWe will define a buildout template used by the recipe:>>> buildout_template = """
... [buildout]
... develop = %(dev)s
... parts = sample
...
... [sample]
... recipe = rod.recipe.appengine
... eggs = foo.bar
... packages =
...     bazpkg
...     tinypkg
... server-script = dev_appserver
... zip-packages = False
... exclude = .*foo/bar/test.*$
... url = http://googleappengine.googlecode.com/files/google_appengine_1.5.0.zip
... """We’ll start by creating a buildout:>>> import os.path
>>> import rod.recipe.appengine.tests as tests
>>> egg_src = os.path.join(os.path.split(tests.__file__)[0], 'foo.bar')
>>> baz_pkg = os.path.join(os.path.split(tests.__file__)[0], 'bazpkg')
>>> tiny_pkg = os.path.join(os.path.split(tests.__file__)[0], 'tinypkg')
>>> write('buildout.cfg', buildout_template %
...       {'dev': egg_src+' '+baz_pkg+' '+tiny_pkg})Running the buildout gives us:>>> print system(buildout)
Develop: '.../tests/foo.bar'
Develop: '.../tests/bazpkg'
Develop: '.../tests/tinypkg'
Installing sample.
rod.recipe.appengine: downloading Google App Engine distribution...
Generated script '/sample-buildout/bin/dev_appserver'.
Generated script '/sample-buildout/bin/appcfg'.This will generate some scripts:>>> ls('bin')
-  appcfg
-  buildout
-  dev_appserverAnd now we try to run the appserver script:>>> print system(os.path.join('bin', 'dev_appserver'), '-h')
Runs a development application server for an application.
...There should be a configuration script in bin as well:>>> print system(os.path.join('bin', 'appcfg'))
Usage: appcfg [options] <action>
...Let’s see if the ‘tests’ directory has been excluded:>>> l = os.listdir(os.sep.join(['parts', 'sample', 'foo', 'bar']))
>>> assert 'tests' not in lThere should be a baz package within our application directory:>>> assert 'baz' in os.listdir(os.sep.join(['parts', 'sample']))Let’s define another buildout template used by the recipe:>>> buildout_template = """
... [buildout]
... develop = %(dev)s
... parts = second_sample
...
... [second_sample]
... recipe = rod.recipe.appengine
... eggs = foo.bar
... use_setuptools_pkg_resources = True
... packages =
...     bazpkg
...     tinypkg
... patch = %(patch)s
... patch-options = -p1
... zip-packages = False
... exclude = tests
... url = http://googleappengine.googlecode.com/files/google_appengine_1.5.0.zip
... """We’ll start by creating a buildout:>>> import os.path
>>> import rod.recipe.appengine.tests as tests
>>> egg_src = os.path.join(os.path.split(tests.__file__)[0], 'foo.bar')
>>> baz_pkg = os.path.join(os.path.split(tests.__file__)[0], 'bazpkg')
>>> tiny_pkg = os.path.join(os.path.split(tests.__file__)[0], 'tinypkg')
>>> patch = os.path.join(os.path.split(tests.__file__)[0], 'patch.diff')
>>> write('buildout.cfg', buildout_template %
...       {'dev': egg_src+' '+baz_pkg+' '+tiny_pkg, 'patch': patch})Running the buildout gives us:>>> output = system(buildout)
>>> if output.endswith(
...     "patching file lib/patched/patched.txt\n"): True
... else: print output
TrueAnd now we try to run the appserver script:>>> print system(os.path.join('bin', 'second_sample'), '-h')
Runs a development application server for an application.
...Let’s have a look if all dependent packages are copied into our application
directory:>>> os.path.isfile(os.path.join('parts', 'second_sample', 'tinymodule.py'))
True
>>> os.path.isdir(os.path.join('parts', 'second_sample', 'baz'))
TrueSetuptool’s original pkg_resources.py file should be copied into our app
directory:>>> pkg_resources = os.path.join(
...     'parts', 'second_sample', 'pkg_resources.py')
>>> os.path.isfile(pkg_resources)
True
>>> pkg_resources_file = open(pkg_resources, "r")
>>> pkg_resources_file.read().startswith('def _dummy_func')
False
>>> pkg_resources_file.close()We’ve configured the recipe to patch the SDK’s source tree:>>> gae_sdk_root = os.path.join('parts', 'google_appengine')
>>> patched_dir = os.listdir(os.path.join(gae_sdk_root, 'lib'))
>>> patched_file = open(
...     os.path.join(gae_sdk_root, 'google', 'appengine', 'tools',
...                  'dev_appserver.py')).read()[:1300]
>>> 'patched' in patched_dir
True
>>> '# This file is patched by the patch command.' in patched_file
TrueYou can also add some extra script:>>> buildout_template = """
... [buildout]
... develop = %(dev)s
... parts = script_sample
...
... [script_sample]
... recipe = rod.recipe.appengine
... eggs = foo.bar
... use_setuptools_pkg_resources = True
... packages =
...     bazpkg
...     tinypkg
... zip-packages = False
... exclude = tests
... url = http://googleappengine.googlecode.com/files/google_appengine_1.5.0.zip
... entry-points = manage=django.core:execute_manager
... initialization = import settings
... arguments = settings
... # your script may need some extra-paths
... extra-paths =
...    /some/extra/path
...    ${buildout:parts-directory}/google_appengine/lib/simplejson
... """We’ll start by creating a buildout:>>> import os.path
>>> import rod.recipe.appengine.tests as tests
>>> egg_src = os.path.join(os.path.split(tests.__file__)[0], 'foo.bar')
>>> baz_pkg = os.path.join(os.path.split(tests.__file__)[0], 'bazpkg')
>>> tiny_pkg = os.path.join(os.path.split(tests.__file__)[0], 'tinypkg')
>>> write('buildout.cfg', buildout_template %
...       {'dev': egg_src+' '+baz_pkg+' '+tiny_pkg})>>> print 'Start...', system(buildout)
Start...
Generated script '/sample-buildout/bin/manage'...Then you get a script:>>> cat('bin', 'manage')
#!...python...
import sys
sys.path[0:0] = [
    '/some/extra/path',
    '/sample-buildout/parts/google_appengine/lib/simplejson',
    '/sample-buildout/parts/google_appengine',
    ]
<BLANKLINE>
<BLANKLINE>
import settings
<BLANKLINE>
import django.core
<BLANKLINE>
if __name__ == '__main__':
    django.core.execute_manager(settings)Changes2.0.6 2014-12-31use_setuptools_pkg_resources support for pkg_resources as a package
and the setuptools zip is unpacked during installation [jjmurre]2.0.5 2014-12-30pkg_resources has changed from a module to a package which broke
the ‘use_setuptools_pkg_resources’ option. Now the package is copied if
it is a package. [jjmurre]2.0.4 2014-09-04Added ‘ignore-packages’ configuration, to exclude problematic packages [jjmurre]2.0.3 2013-06-17Fix for issue 18/20/21: appcfg should not use the ‘appserver-run-file’ config. variable.2.0.2 2013-06-18Added ‘appserver-script-name’ configuration, to accomodate alternative
appserver scripts names (some versions in the 1.7.x range used
dev_appserver2.py). [jjmurre]Added ‘appserver-run-file’ configurations, because Google changed this
name somewhere in the 1.7.x version range. [jjmurre]Added ‘symlink-gae-runtime’. Throught this switch a symlink to
the GAE runtime script ‘_python_runtime.py’ can be made in de
buildout bin directory. This symlink is needed, because the
developments server had been thouroughly rewritten by Google. [jjmurre]2.0.1 2013-02-21Made dev_appserver configurable (dev_appserver/dev_appserver2) [attilaolah]2.0.0 2011-07-01Adds support for zc.recipe.eggs options such as entry-points and
extra-paths.1.7.0 2011-05-16Adds support for regular expression excludes.Minor refactoring and clean-up.1.6.2 2010-04-18Fixes an issue where symlink fails on platforms other than UNIX.1.6.1 2010-04-03Fixes an issue where the patch option can’t be used without patch-options.1.6.0 2010-04-03New option to specify a patch file for modifying the Google App Engine
SDK source tree.1.5.1 2010-03-27Fixes an issue where setuptools wasn’t found.Distribute can be used as a replacement for setuptools.Added credits for Attila Olah.1.5.0 2010-03-27Adds option to copy setuptool’s original pkg_resources.py file into app
directory rather than writing a dummy stub.1.4.1 2010-01-18Fixes an issue where egg contents which are just single modules aren’t
copied into the project.http://code.google.com/p/rodrecipes/source/detail?r=141.4.0 2009-08-26Added server-script option.Tests added.1.3.1 2009-07-15Fixed issue when copying egg contents.1.3.0 2009-07-04Added options zip-packages and exclude.1.2.1 2009-07-03Uses a much better method for excluding optional c extensions and compiled
modules.A step forward in platform independence.1.2.0 2009-06-24Creates appcfg script.1.1.1 2009-06-07Makes symbolic links for application files.Downloads stay untouched.1.1.0 2009-04-08Mostly rewritten.Installs google appengine as part.Adding dummy pkg_resources module to handle namespace package relicts.Tests added.Ready for Google App Engine SDK 1.2.01.0.0b5 2009-01-20Requires Google App Engine SDK 1.1.81.0.0b4 2008-09-04Create and use PROJECT_HOME/var to place temporary project files like
data base files.1.0.0b3 2008-09-02Copy package contents to temporary library directory.1.0.0b2 2008-09-02Installs the whole distribution in the parts directory now. So it is ready
to test and deploy.1.0.0b1 2008-09-01First beta release.