(#21376) Sequence BlockExpressions without nesting

When merging Puppet::Resource::Type instances the code attribute was merged in
a way that caused a deep tree of BlockExpressions. This could cause stack
overflows when evaluating this tree, since evaluation is recursive. This
changes the merge method so that they are sequenced as a flat array.
(Maint) Run tests from installed packages

This creates the basic structure for running tests from packages. It
includes a Rakefile for fetching the appropriate artifacts and executing
systest, the setup steps for installing the repos and packages, and a
basic test that makes sure that puppet is installed.

  bundle install
  bundle exec rake test SHA=COMMITISH

where COMMITISH corresponds to the SHA of a package built via the
puppetlabs-packaging job.
(#21320) Use sleep in the daemon loop instead of select

This fixes a bug on Ruby < 2.0 where the daemon might sleep for a very long
period after being interrupted with a signal. The implementation of select on
these ruby versions has an improper cast which resulted in a corrupted value
given to select on 64 bit systems. It is patched in Ruby 2.0 in
https://github.com/ruby/ruby/commit/b10c3136b0163536a679b10fecbc2992f3b5fa64 .

Sleep does not attempt to restart after being interrupted and does not have this bug.
(#21043) Use ruby.exe instead of rubyw.exe

Previously, if you set a custom runinterval setting in puppet.conf, puppet
running on ruby 1.9 would ignore it, whereas it would work in ruby 1.8.

The root cause is due to http://bugs.ruby-lang.org/issues/7239. In ruby 1.9,
rubyw.exe is unable to read the stdout of any child process using ruby's
built-in process methods, e.g. %x. This affects us, because the windows
service, running as rubyw.exe, executes:

    `%x{ puppet agent --configprint runinterval}`

to obtain the current runinterval setting.

The bug is fixed in ruby 2.0, so for now, we just switch to using
`ruby.exe` instead of `rubyw.exe`.
(Maint) Cleanup specs

This cleans up the specs to make them a little more expressive of the
behavior of the fqdn_rand function's real functionality. One surprising
result that came out of this was that the tests tried to assert that the
fqdn_rand function was callable without any arguments. However the test
for this passed simply because it asserted that a particular error was
*not* raised, when in fact a different exception that expressed what it
was trying to catch (ArgumentError vs. Puppet::ParserError). The docs
indicate that calling with no arguments is not actually supported and so
that test has been removed.
(#21264) Update rgen dependency to 0.6.5

There is an error in rgen < 0.6.5 that causes it to incorrectly handle
Array#to_str via its monkey patching of Array#method_missing. This adds
a check for this incorrect behavior and updated our Gemfile dependency
on rgen. The package dependencies are already specified in a manner that
will pick up 0.6.5 once it is packaged.
(#20607) support symbolic file mode in Puppet::Util#replace_file

In order to fix #20607 (where incorrect 'settings specified' file
permissions were crashing P::U#replace_file), this patch enhance
P::U#replace_file default_mode behavior.

Now P::U#replace_file supports symbolic file mode, along with
octal numerical string, and pure octal integers. It also now fails
if the given mode is incorrect.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
(maint) Includes missing spec_helper in three specs.

spec/spec_helper was missing from three of the specs. This showed up
because spec/unit/pops/containment_spec.rb just happened to come first
in the order of specs to be executed on our Solaris build, and began
causing failures because puppet/pops was being loaded prior to
spec_helper and puppet.  It was also causing rspec/mocha setup and
teardown failures, presumably because of rspec configuration being
called after this initial spec was loaded.

This commit ensures that each of these specs requires spec_helper first
so that spec and the puppet environment initialize properly regardless
of whether these specs run first or come later in the spec ordering.
Deprecate sending YAML over the network

I'm not sure if this covers all the corners. I didn't want to do it in the
Format itself because that can be used in other places.
(#21427) Send facts to master as pson

This changes the format of facts being sent to the master from
b64_zlib_yaml (base 64 encoded compressed yaml) to pson, in preparation
for deprecating all YAML sent over the network. This should be backwards
compatible as facts are tagged with the format and the master already
supports pson.
(#21427) Issue deprecation warning for b64_zlib_yaml
(#21427) Improve deprecation warning for yaml params

Previously we were not informed what parameter and value was being
transmitted as YAML, which made debugging and correcting the problem
nearly impossible. This outputs the offending request parameter and
value as a debug message to aid in fixing the issue.
(Maint) Cleanup do_save specs

This replaces the excessive mocking in the do_save specs with using
actual implementations and removes mocking on the class under test. By
doing so there are many fewer tests required and the remaining tests are
more directly related to externally observable behavior. This has only
been done for the do_save method.
(#21427) Respect Accept header for save

The REST save action will now respect the formats that are requested in
the Accept header. In order to retain backwards compatibility for
clients that have never been required to specify an Accept before this
it will also default to :yaml when no Accept is provided. If an Accept
is provided but none of the wanted formats are suitable an error is
raised, just like find and search.

In order to report the status of an unkown Accept header correctly this
also adds the capability of raising web related exceptions that contain
the HTTP response code that needs to be used.
(#21427) Respect Accept header for destroy

The REST destroy action will now respect the formats that are requested in
the Accept header. In order to retain backwards compatibility for
clients that have never been required to specify an Accept before this
it will also default to :yaml when no Accept is provided. If an Accept
is provided but none of the wanted formats are suitable an error is
raised, just like find and search.
(#21427) Remove duplicate handling of accept headers

When the save and destroy actions started respecting the accept header
we duplicated how accept headers were handled for selecting the response
format. This removes that duplication and in putting specific tests in
place, cleans up the tests to work mainly through the public interfaces
as well as putting in place an HTTP error handling mechanism that allows
the request handlers to more specifically control the HTTP responses.
Invalid Accept headers will now cause a 406 response instead of the
previous 400.

Since all of this is tested with the memory terminus, it needed to be
expanded to also handle search and head actions.
(#21427) Deprecate YAML over the network

Issue a deprecation warning whenever YAML is encoded or decoded in the
http handler.
(#21427) Submit reports in PSON format

This changes the agent to submit reports in PSON. An effect of this
change is that reports will not be able to submit to masters that do not
have the changes for CVE-2013-3567 which allow the master to parse
PSON formatted reports.
(Maint) Remove stubbing from query_string tests

The tests for the Puppet::Indirector::Request#query_string method were
relying on stubbing methods on the class under test. There is no need
for that since the required data is trivially passed in via the
constructor. This replaces the stubbing with simply constructing the
correct object to test.
(Maint) Test that the query string is parsable

Rather than relying on all sorts of string checks, it is better to check
that a parameter parsing library is able to parse the generated string
back into the expected data. This changes the tests to use CGI::parse to
parse the query_string and then can check that the resulting hash has
the data that is expected.