Package: cgcloud-jenkins
The CGCloud Jenkins project contains the roles for running a distributed
continuous integration environment in EC2 with one Jenkins master VM and
multiple slave VMs. A Jenkins slave is a machine that the master delegates
builds to. Slaves are launched on demand and are shutdown after a certain
amount of idle time. The different slave roles are blueprints for setting up a
slave VM that has the necessary prerequisites for running a particular Jenkins
build.QuickstartActivate the virtualenv cgcloud was installed in and installcgcloud-jenkins:::

   cd
   virtualenv cgcloud
   source cgcloud/bin/activate
   pip install cgcloud-jenkins
   export CGCLOUD_PLUGINS="cgcloud.jenkins:$CGCLOUD_PLUGINS"If you getDistributionNotFound: No distributions matching the version forcgcloud-jenkins, try runningpip install--precgcloud-jenkins.Runningcgcloudlist-rolesshould now list the additional roles defined in
the plugin:...
jenkins-master
ubuntu-lucid-genetorrent-jenkins-slave
ubuntu-precise-genetorrent-jenkins-slave
ubuntu-saucy-genetorrent-jenkins-slave
ubuntu-trusty-genetorrent-jenkins-slave
centos5-genetorrent-jenkins-slave
centos6-genetorrent-jenkins-slave
fedora19-genetorrent-jenkins-slave
fedora20-genetorrent-jenkins-slave
ubuntu-lucid-generic-jenkins-slave
ubuntu-precise-generic-jenkins-slave
ubuntu-saucy-generic-jenkins-slave
ubuntu-trusty-generic-jenkins-slave
centos5-generic-jenkins-slave
centos6-generic-jenkins-slave
fedora19-generic-jenkins-slave
fedora20-generic-jenkins-slave
centos5-rpmbuild-jenkins-slave
centos6-rpmbuild-jenkins-slave
load-test-box
data-browser-jenkins-slaveMaster And Slave RolesThe plugin defines a role for the master (jenkins-master) and various slave
roles for running builds for certain building CGL projects. There are also a
bunch of generic slaves that are not customized for a particular project.The master (jenkins-master) is a long-running box that hosts the Jenkins
web application. The Jenkins installation (code and data) is cordoned off in
the home directory of a separatejenkinsuser. That home directory actually
resides on a secondary EBS volume whose life cycle is independent from that of
the master box, i.e. VM instance. This allows us to update the OS of the master
without having to setup Jenkins from scratch every time we do so.The remaining roles define the Jenkins slaves. A Jenkins slave is a
short-running box with which the master establishes an SSH connection for the
purpose of triggering aremote build. The CGCLoud Jenkins plugin (this
project) is used to create the VM images and register them with the master such
that the master can launch a slave instance when needed to run a remote build
on the platform provided by the slave.JenkinsJenkins is a continuous integration server/web applicaton running on thejenkins-master. Jenkins uses so calledprojectsthat define where to get
the source, how to build and test the source and which build artifacts to
archive. Builds can be run automatically whenever a push is made, on a fixed
schedule or manually. Builds are executed by an agent. Agents can run locally
on the Jenkins master or remotely on one or more slaves. Jenkins uses its own
plugin system to extend and modify the default behavior. We use the EC2 plugin
which allows us to create slaves on demand in EC2 from images created by
cgcloud in conjunction with this project. Mind the distinction between CGCloud
Jenkins which is plugs into CGCLoud and the hundreds of plugins that extend
Jenkins.The Jenkins web UI can always be accessed by tunneling port 8080 through SSH.
Runningcgcloud ssh jenkins-mastersets up the necessary port forwarding.
Authorization and authentication in Jenkins itself is disabled on a fresh
instance but can be enabled and further customized using Jenkins plugins. Note:
Anyone with SSH access to the master can access Jenkins and do anything with it.Tutorial: Creating a Continuous Integration EnvironmentIn this tutorial we’ll create a continuous integration environment consisting
of a Jenkins master and several slaves. The tutorial assumes that you completed
the Quickstart section of the CGCloud README.Creating The MasterCreate the Jenkins master instance:cgcloud create jenkins-masterAs a test, SSH into the master as the administrative user:cgcloud ssh -a jenkins-master
exitThe administrative user hassudoprivileges. Its name varies from platform
to platform butcgcloudkeeps track of that for you. For yet another test,
SSH into the master as thejenkinsuser:cgcloud ssh jenkins-masterThis is the user that the Jenkins server runs as.Next, create an image of the master such that you can always recreate a 100%
identical clone:cgcloud stop jenkins-master
cgcloud image jenkins-master
cgcloud terminate jenkins-master
cgcloud recreate jenkins-masterThe first command is necessary to stop the master because only a stopped
instance can be imaged. Theimagecommand creates the actual AMI image. Theterminatecommand disposes of the instance. This will delete the/partition while leaving the/var/lib/jenkinspartition around. The latter
is stored on a separate EBS volume calledjenkins-data. In other words, theterminatecommand leaves us with two things: 1) the AMI for a master box
and 2) the Jenkins data volume. Therecreatecommand then creates a new
instance from the most recently created imageandattaches thejenkins-datavolume that instance.Creating The SlavesOpen a new shell window and create the first slave:cgcloud list-roles
cgcloud create docker-jenkins-slaveSSH into it:cgcloud ssh -a docker-jenkins-slaveNotice thatThe admin user has sudo rights:sudo whoamiThe builds directory in the Jenkins user’s home is symbolically linked to
ephemeral storage:sudo ls -l ~jenkinsgit and docker are installed:git --version
docker --version
exitNow stop, image and terminate the box:cgcloud stop docker-jenkins-slave
cgcloud image docker-jenkins-slave
cgcloud terminate docker-jenkins-slaveFinally, register all slaves with the master:cgcloud register-slaves jenkins-master docker-jenkins-slaveTheregister-slavescommand adds a section to Jenkins’ config.xml defines
how to spawn an EC2 instance ofdocker-jenkins-slavefrom the AMI we just
created. The slave description also associates the slave with the labeldocker. If a project definition requests to be run on slaves labelleddocker, an instance will be created from the AMI. Once the instance is up,
the Jenkins master will launch the agent on via SSH. Finally, the master will
ask the agent to run a build for that project. If a slave labelleddockeralready exists, it will be used instead of creating a new one. You can
customize how may concurrent builds run on each slave by increasing the number
of agents running on a slave. By default only one slave per role will be
launched but you can configure Jenkins to launch more than one if the queue
contains multiple builds for a given label.