Merge pull request #148 from MosesMendoza/fix_signing_and_tarring

Fix signing and tarring
Add support for the build manager

This adds a number of `pe:jenkins:managed_*` and `pl:jenkins:managed_*
tasks to interact with the build manager. These jobs are wrappers
around the non-managed tasks of similar name which initialize the
build manager for a new build, and then call the underlying task with
some extra parameters set in order to ensure the various packaging
runs know they are managed.

It also modifies the `pl:deb`, `pl:mock`, `pl:gem`, `pl:dmg`,
`pl:tar`, and `pe:local_sles` tasks so that they will check in to the
build manager on completion if a managed build is running.
Add a module for rake task object utilities

This is the beginning of hopefully providing some logic to the madness of the
utilities in the packaging repo.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Rename 00_utils.rake to 01_utils.rake

We will be adding a monkey_patches.rake file, and to ensure it gets loaded
first, we have to display the long-standing initial utils file, 00_utils.rake.
00_utils.rake becomes 01_utils.rake to make way for 00_monkey_patches.rake.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Add 00_monkey_patches.rake with Rake::Task#unshift method

This commit adds 00_monkey_patches.rake, which will be a file containing any
monkey patches we need. Initially, this contains Rake::Task#unshift, which is a
method that is based of the existing Rake::Task#enhance method. Rake task
objects contain an @actions variable, which is an array of blocks that are
executed when the #execute method is called. These blocks are normally created
by the common `task :name` syntax. Under the hood, Rake calls #enhance on the
task, which builds the task by appending a block passed as an argument to
tasks workflow, which is not to append to execution blocks, but rather prepend.
That is, we want a block to fire off before a tasks existing actions. This is
what the #unshift method provides. Similar to Array#unshift, this method
inserts a block at the beginning of the array of blocks to be executed by a
rake task.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Update spec tests for renamed 01_utils.rake

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Add a test for Rake::Task#unshift

This commit tests that the monkey patched #unshift method does indeed insert an
execution block before a Rake task's existing blocks, so that when that task is
executed the block passed with #unshift is executed first.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Create BuildManager class and move post methods into it

This commit creates a TaskOrchestration::BuildManager class, which will
encompass the behavior of posting start and result class to the build server.
It has instance variables for the job_id, timeout, count, ref, downstream job,
and management server. This is another step toward moving the packaging repo's
functionality into discreet classes. This commit also slightly refactors
@branan's original post methods to behave from within the class instead of as
global methods availble to rake.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Add ENV overrides for @build.managed and @build.management_server

These will be useful as command line flags to a package build,
so we enable setting them via ENV['MANAGED'] and ENV['MANAGEMENT_SERVER'].

This way we can test the management via command line without modifying build
parameters.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Update check_var to return the variable being checked

This makes check_var slightly more useful for inline assignments. This is fully
backwards compatible.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
temporarily disable curl
Temporarily comment inaccessible methods
Add support for modifying tasks on the fly for build manager

Modify existing Rake tasks with behavior to support managed builds Use
monkey-patched #unshift method and existing #enhance method to enable
inserting actions into already defined rake tasks on the fly.

For core tasks (e.g. pl:deb, package:tar, etc) we use #enhance to add an
additional execution block after the existing tasks, containing a status post
to the build server. Each individual task (e.g. deb cow build) has to post a
status, which is why we're adding at this level. Once the task finishes it posts to the build manager that it has completed.

For aggregate-style task, e.g. "pl:jenkins:uber_build" we prepend to the
existing execution blocks with one which notifies the build manager that a
build has started. Prepending to existing execution blocks offers the ability
to retain all existing task behavior including order in dependencies.

The downside of this approach over @branan's approach is that there is no
explicit failure state from a failed package task, because if a task throws an
exception in one block, Rake will not continue with the remaining blocks. This
means a failure will always mean a timeout, because the core task will never post a status if it fails.

The upside of this approach is that it is much cleaner (for varying definitions
of clean) in that does not add more tasks or clutter to the packaging repo, and
is largely transparent to all existing users. Just add MANAGED=true and the
rest is taken care of for you.
Add task.count to params to post_managed_build_start

This commit removes @count as an instance variable of the BuildManager class,
since its only ever used as a parameter to the managed build start method.
Instead we just pass it in as part of the call to the method. This will rely on
"count" living somewhere else - we're going to add it to the task itself, by
monkey-patching @task_count as a Rake::Task instance variable.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Monkey patch in @count into Rake::Task

This allows us to keep track of how many different tasks we anticipate a task
will kick off. This will allow us to know, for example, how many internal
"pl:deb" calls a pl:jenkins:deb_all call will make, which will then allow us to
track progress of the individual tasks from the build manager. This commit
includes a basic test to ensure instances of Rake::Task have accessors for
@count.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Set counts on the task objects, remove old make_managed_task methods

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Add a target_count utility

Just returns the number of targets we can expect given an aggregate build task.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Set counts on the task objects, remove old make_managed_task methods

We're only going to support build coordination for uber_build tasks, for pe and
foss. This dramatically simplifies the logic needed to dynamically prepend
aggegrate task execution with a call to start a build manager watcher, because
the uber_build tasks are aggregates of aggregates. If we supported deb_all and
mock_all, we'd have to figure out some way to selectively prepend based on
which task we're going to eventually use, which is unreliable.

Signed-off-by: Moses Mendoza <moses@puppetlabs.com>
Remove remaining tasks from original implementation of build manager
Update README.md