edx/configuration — pull request #409 by wedaly (2013-11-04 15:39:54+00:00)
Title: Jenkins worker refactor

Refactor of Jenkins worker role:

1) Use rbenv and edxlocal roles. This required a slight change to the rbenv role: when using the rbenv environment from the shell, it defaults to using rake version 0.9.2 (installed with ruby in the rbenv). The Ansible script removes that version and installs a later version (10.0.3). This is the only way I've found to prevent rbenv from preferring the system installation to the gem installation of rake.

2) Remove upstart scripts that set up mongo/python/jenkins workspace in ephemeral storage. This fixes a race condition between the upstart scripts and Jenkins jobs that would sometimes cause build failures.

3) Create a directory of wheel archives at AMI creation, which Jenkins jobs can use to create virtualenvs with pre-installed requirements. This is less flexible than the previous implementation (which downloaded .egg files from S3), but much faster and more reliable.

If we later upload wheel files to S3, it should be easy to update the "wheel_venv.sh" script to download the archive instead of assuming it's part of the AMI.

Suggested reviewer: @feanil