(#3910) Include environment in catalog

This adds an environment field to a catalog, so that the puppet master can
tell agents what environment it used to compile the catalog and what it
expects them to use.
Remove unused clear method in configurer
(#3910) Make puppet agent use environment specified in catalog

If there is an environment specified in the catalog to be applied (i.e. an
external node classifier was used and gave an environment for this node) the
puppet agent will switch to that environment.

When switching environment, the agent needs to get facts and plugins again
using the new environment, and then try fetching the catalog again using the
new facts, since this may change the catalog. It loops in this process several
times, until the environment doesn't change. If it fails to stabilize in a
limited number of tries it will raise an error and abort the run.

This change also means that the prerun command runs after fetching catalog but before
applying. It previously ran before fetching the catalog.
(#3910) Acceptance tests for environments and ENC

Added acceptance tests around the setting of environments by an agent and an
ENC. Verify that pluginsync and filebucket requests use the correct
environment.
Documenting missing --noop option for puppet apply
Revert "Provide a helpful error message when name resolution fails"

This reverts commit aae3a58b7d241b33b7a12ff2aadafa2db3b4904d.
Merge pull request #569 from pcarlisle/ticket/master/3910-server-authoritative

Ticket/master/3910 server authoritative
Merge pull request #570 from francois/master

Document missing --noop option for puppet apply
Add a feature to detect RDoc 1.0.1 being present on the system.

In practice the Puppet manifest documentation generator only works with the
version of RDoc that ships with Ruby 1.8 - version 1.0.1, which was identical
in 1.8.5 ad 1.8.7.

Previously, because it was a core library, we just assumed it was present; now
we need to test conditionally for it being there *and* the correct version.

That allows us to have a nicer failure mode as we work through rdoc use, to
clearly indicate to users which version we have.  As we support newer
versions, additional features can better identify what is available.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Fix 4-space indentation in puppet/util/rdoc.rb

There are no functional changes to this commit, just whitespace changes.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Document and cleanly fail when unsupported RDoc is used.

Update the documentation and code to fail cleanly when a supported version of
RDoc is not available.

This does not yet tackle loading an RDoc version specific implementation of
the process, only a clean failure when we can't cope.  In future this should
be replaced by some dynamic loading process based on the supported RDoc
version, if we retain the tool.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Ruby 1.9.3 is strict about block argument counts.

Unlike earlier versions, Ruby 1.9 is strict about matching argument counts in
blocks with their uses.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Ruby 1.9.3 has a different error when `require` fails.

The text of the error message when load fails has changed, resulting in the
test failing.  This adapts that to catch the different versions, allowing this
to pass in all cases.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Simplify a YAML report test to work with Ruby 1.9

The order of invocation of local method definition and the parent `inherited`
hook changed between 1.8 and 1.9, resulting in the hack that works around a
development check in this test stopping working.

None of that ceremony is actually needed, though, since we can totally just
use an instance of the actual class we are testing.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Don't cache check for `cdrom` source in the apt provider.

Ruby 1.9 warned about access to a class variable in providers, because their
semantics are almost never what users expect.

However, the use of a class variable in the apt provider was very strange: it
wanted to check if a cdrom source was specified, and fail if it was present,
to avoid a situation where apt would prompt for a CD to be inserted and hang
forever.

Since that is just as much a problem on the second package install as on the
first, it makes much more sense to perform this sanity check and fail before
every install - and the overall cost is relatively low.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Avoid class level variables, as they are not helpful.

Ruby 1.9.3 highlights some of the strange and confusing problems around class
level variables in Ruby; they tend to look and behave much more like global
state than people imagine, and lead down the road to trouble.

This migrates away from them to other mechanisms better fitted to solving the
problem in the Ruby language - and, typically, in ways that are much more
portable between the different Ruby interpreter implementations.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Revert "Disable replace_file on Windows"

This reverts commit 4272d1f7674911c15da3ff6516d5827646a6bda6.

To quote:

    Our current approach of setting owner/mode in the Puppet way risks
    losing any extra permissions beyond owner/group/everyone, which is
    significant enough that it is better to simply not support this
    operation until we have a better solution. replace_file isn't used in
    any code which actually runs on Windows at the moment, so this is
    a benign change.

...except that it isn't so benign as all that: various other parts of the code
open-code the same method, poorly, and with various different bugs.  They will
all, without question, lose permissions in exactly the same way on Windows.

At least by routing them through a central path we can start to unify their
behaviour and get to a place where fixing one point in the code fixes all the
faulty permission transfers that we previously had.

We win an overall net reduction in bug count, and an increase in
predictability of bugs, from this change.
Replace Puppet::Util::FileLocking with atomic `replace_file`

The Puppet::Util::FileLocking module had substantial complexity in aide of
allowing careful, synchronized updating of individual files.  It was super
careful, and wanted to be absolutely sure that it didn't have concurrent reads
and writes going on, so it:

 * used per-file Ruby level locks to avoid multiple threads accessing the file
   at the same time.
 * used file-system level cooperative file locking on the target file to avoid
   multiple processes seeing concurrent updates.
 * used a temporary file to avoid rewriting the target file in place, so there
   was no "partial file read" window.
 * carefully used portable constructs to achieve locking rather than
   OS-specific kernel level exclusions.

Of course, some of those claims were documented but no longer valid - using a
temporary file *and* cooperative file locking don't play nice, and the
temporary file bit had dropped from the actual implementation forever ago.

It also did nothing to satisfy any of the other interesting conditions around
files: if multiple concurrent writers began, there were no assurances beyond
chance in the OS process scheduler which would win and replace the file.

In practice all that was really wanted were some very, very simple semantics:

Any process or thread can read the file at any time, and will get a complete
and correct version of the file.

Any process or thread can write at any time, provided that the read constraint
is satisfied.

It turns out that we recently built a method with these exact semantics,
`replace_file`, which will perform an atomic replacement using rename, but
only if the new file content is safely and completely stored on disk.

That satisfies all current semantics: any process can read at any time, and
will either get the complete previous version, or the complete new version,
depending on how it schedules around the (POSIX defined atomic) rename.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Fix terminus testing for Ruby 1.9

The order of evaluation of abstract classes (Class.new) changed in Ruby 1.9,
resulting in the hacks here that worked around the "must be bound to a
constant" rules in the indirector starting to fail.

Technically, though, we don't actually care: we can define a whole class stack
properly, binding everything the way the system expects, and then clean it up
afterwards.

Less stubbing, better tests, and clearer logic, all of which are pretty nice.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Handle YAML load errors from the new Psych YAML parser.

Earlier versions of Ruby used the syck library to parse YAML; during 1.9 this
moved to the Psych parser based on libyaml.  This brings YAML 1.1 compliance
to the library, and generally moves in a forward direction.

It also makes some radical changes to the failure modes, since Psych is much
more forgiving of some YAML constructs that would previously block the heck
up.

It also, in Ruby tradition, does absolutely nothing to abstract over the
implementation: Ruby can use either Psych or Syck and will raise different
exceptions on failure in both cases.

Worse, Psych exceptions are not derived from StandardError, so
Psych::SyntaxError will actually bypass almost any normal attempt to trap the
error, and will have wildly different details to the Syck errors.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>