(QENG-188) Allow foss runs to use service scripts, take 2

Prior to this commit, any run of beaker with a non-pe master that
attempted to stand up a master with a particular configuration in order
to test against it using the with_puppet_running_on() helper would
always stand up a webrick master by executing `puppet master <args>` and
then later stop it with `kill`.

The platform team needs to be able to run acceptance suites in which the
puppetmaster is started/stopped using package provided init scripts, or
by restarting apache if a passenger puppetmaster package is installed.

This PR takes the approach of enhancing Beaker::Host with a few query
methods controlled by host properties, either set in the hosts.cfg, or
directly on a master Beaker::Host instance during the execution of a
setup step.  The added properties are:

 * 'use-service' : if true, service scripts will be used instead of
manually starting a webrick master
 * 'passenger'   : if true, indicates a passenger package has been
installed, and by default graceful service restarts will be used
 * 'puppetservice' : the 'puppetservice' property has been added to the
Beaker::Host::Unix.foss_defaults with the value of 'puppetmaster'.  It
should be set to the appropriate service script name for passenger if a
passenger package has been installed.
 * 'graceful-restarts' : can be set false if you want to stop/start with
service scripts rather than use graceful restarts when running with
passenger

The Beaker::Host#uses_passenger! call may be used to set a host
appropriately for passenger.

The graceful restarts facility of the dsl helper's bounce method assumes
apache2 was used for passenger and that apach2ctl is available.

Any existing test suite which does not 'opt-in' to any of these service
host changes by specifying any of the above properties should not be
effected by this commit.  The exception to this is that PE runs will use
graceful restarts automatically with this commit.

There is also a spec related change to the FakeHost helper, which
previously was a stub pretending to provide Beaker::Host like facilities
and recording commands for review.  This became unwieldy when the
various query methods for use_service_scripts? is_using_passenger? and
so forth were added, and the FakeHost has been updated to instead use
Beaker::Host.create to provide a host instance that is then extended
with the MockedExec module stubbing the exec facilities for testing.
(QENG-1032) Fix for tests without a master host

If a master role is not defined in the current test's config, then
setting a default node will fail. The code is expecting "hosts" to be a
list, but it is actually a hash. This commit fixes this issue and
adds a new spec test to validate a non-master single node is set as
default.
(QENG-1012) (gh-143) beaker scp moving data one byte at a time

- force chunk_size during transfer
- print more debug metrics so that we can monitor how quickly files are
  moved
(maint) formatting

This removes trailing whitespaces in install_utils.rb
(QENG-1037) Install from git should accept depth

When installing from larger repositories, being able to specify depth can cut
the time of git checkout in half. This becomes especially helpful when you are
checking out multiple repositories.
Due to the older git on some of the templates, this provides the older
--branch name --depth 1 commands, which means when using depth, one should
ensure the rev passed is a branch and not a single commit. Alternatively one
can add depth_branch => 'name' to repository and have that used instead of rev.
Merge pull request #384 from jpartlow/issue/master/qeng-188-better-foss-service-restarts

(QENG-188) Allow foss runs to use service scripts, take 2
Revert "(QENG-188) Allow foss runs to use service scripts, take 2"

This reverts commit 577fce4dcaf4c6879157fd51777d6691eae1496d.
(QENG-188) Allow foss runs to use service scripts

Prior to this commit, any run of beaker with a non-pe master that
attempted to stand up a master with a particular configuration in order
to test against it using the with_puppet_running_on() helper would
always stand up a webrick master by executing `puppet master <args>` and
then later stop it with `kill`.

The platform team needs to be able to run acceptance suites in which the
puppetmaster is started/stopped using package provided init scripts, or
by restarting apache if a passenger puppetmaster package is installed.

This PR takes the approach of enhancing Beaker::Host with a few query
methods controlled by host properties, either set in the hosts.cfg, or
directly on a master Beaker::Host instance during the execution of a
setup step.  The added properties are:

 * 'use-service' : if true, service scripts will be used instead of
manually starting a webrick master
 * 'passenger'   : if true, indicates a passenger package has been
installed, and by default graceful service restarts will be used
 * 'puppetservice' : the 'puppetservice' property has been added to the
Beaker::Host::Unix.foss_defaults with the value of 'puppetmaster'.  It
should be set to the appropriate service script name for passenger if a
passenger package has been installed.
 * 'graceful-restarts' : can be set false if you want to stop/start with
service scripts rather than use graceful restarts when running with
passenger

The Beaker::Host#uses_passenger! call may be used to set a host
appropriately for passenger.

The graceful restarts facility of the dsl helper's bounce method assumes
apache2 was used for passenger and that apach2ctl is available.

Any existing test suite which does not 'opt-in' to any of these service
host changes by specifying any of the above properties should not be
effected by this commit.

There is also a spec related change to the FakeHost helper, which
previously was a stub pretending to provide Beaker::Host like facilities
and recording commands for review.  This became unwieldy when the
various query methods for use_service_scripts? is_using_passenger? and
so forth were added, and the FakeHost has been updated to instead use
Beaker::Host.create to provide a host instance that is then extended
with the MockedExec module stubbing the exec facilities for testing.
Merge pull request #389 from waynr/revert/qeng-188

Revert "(QENG-188) Allow foss runs to use service scripts, take 2"
(MAINT) add 61613 to open port list for ec2 instances

- 61613 needs to be open for MCO to work correctly
(MAINT) aws_sdk support repairs

- vmhostname was being set incorrectly and breaking PE installation
- move instance metadata tag addition to after instance is running
- /etc/hosts generation fixed so that it has entries for all other hosts
  on each individual host
- add agent listen port (8139)
- fix zombie killing by correctly handling volumes that may not exist
  when attempted deletion occurs
(gh-210) Setting PATH, ignored :environment option

- we do not correctly handle adding additional environment variables to
  puppet commands, this patch allows you to add those env vars
Merge pull request #392 from anodelman/maint

(MAINT)(gh-210) series of commits to get beaker smoketest working in ec2
Merge pull request #387 from colinPL/qeng_1032

(QENG-1032) Fix for tests without a master host
Merge pull request #385 from anodelman/scp-repair

(QENG-1012) (gh-143) beaker scp moving data one byte at a time
(QENG-751) Request for Higgs support in installing beaker

- adds install_higgs command
- executes against master node
- downloads pe packages based upon pe_ver/pe_dir
- goes up to the point in the install where human interaction is
  required (notices indication in install log that a link has been
  provided to a web server)
Merge pull request #390 from jpartlow/issue/master/qeng-188-foss-service-restarts-pe-unchanged

(QENG-188) Allow foss runs to use service scripts (pe unchanged)
(QENG-903) Debian platform support for jvm-puppet.

A couple simple fixes for previously unvisited code paths.

Signed-off-by: Wayne <wayne@puppetlabs.com>
(QENG-922) Add `puppet_user` and `puppet_group` DSL methods.

Signed-off-by: Wayne <wayne@puppetlabs.com>
Merge pull request #395 from waynr/feature/QENG-922-puppet-user-group-methods

(QENG-922) Add `puppet_user` and `puppet_group` DSL methods.