Merge branch '2.7.x'

This commit looks like a no-op, but it resolves a conflict for
a commit in 2.7.x that had to do with spec_helper compatibility
w/stdlib and grayskull (#13439)

Conflicts:
	lib/puppet/util/settings.rb
Add a feature to detect RDoc 1.0.1 being present on the system.

In practice the Puppet manifest documentation generator only works with the
version of RDoc that ships with Ruby 1.8 - version 1.0.1, which was identical
in 1.8.5 ad 1.8.7.

Previously, because it was a core library, we just assumed it was present; now
we need to test conditionally for it being there *and* the correct version.

That allows us to have a nicer failure mode as we work through rdoc use, to
clearly indicate to users which version we have.  As we support newer
versions, additional features can better identify what is available.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Fix 4-space indentation in puppet/util/rdoc.rb

There are no functional changes to this commit, just whitespace changes.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Document and cleanly fail when unsupported RDoc is used.

Update the documentation and code to fail cleanly when a supported version of
RDoc is not available.

This does not yet tackle loading an RDoc version specific implementation of
the process, only a clean failure when we can't cope.  In future this should
be replaced by some dynamic loading process based on the supported RDoc
version, if we retain the tool.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Ruby 1.9.3 is strict about block argument counts.

Unlike earlier versions, Ruby 1.9 is strict about matching argument counts in
blocks with their uses.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Ruby 1.9.3 has a different error when `require` fails.

The text of the error message when load fails has changed, resulting in the
test failing.  This adapts that to catch the different versions, allowing this
to pass in all cases.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Simplify a YAML report test to work with Ruby 1.9

The order of invocation of local method definition and the parent `inherited`
hook changed between 1.8 and 1.9, resulting in the hack that works around a
development check in this test stopping working.

None of that ceremony is actually needed, though, since we can totally just
use an instance of the actual class we are testing.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Don't cache check for `cdrom` source in the apt provider.

Ruby 1.9 warned about access to a class variable in providers, because their
semantics are almost never what users expect.

However, the use of a class variable in the apt provider was very strange: it
wanted to check if a cdrom source was specified, and fail if it was present,
to avoid a situation where apt would prompt for a CD to be inserted and hang
forever.

Since that is just as much a problem on the second package install as on the
first, it makes much more sense to perform this sanity check and fail before
every install - and the overall cost is relatively low.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Avoid class level variables, as they are not helpful.

Ruby 1.9.3 highlights some of the strange and confusing problems around class
level variables in Ruby; they tend to look and behave much more like global
state than people imagine, and lead down the road to trouble.

This migrates away from them to other mechanisms better fitted to solving the
problem in the Ruby language - and, typically, in ways that are much more
portable between the different Ruby interpreter implementations.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Revert "Disable replace_file on Windows"

This reverts commit 4272d1f7674911c15da3ff6516d5827646a6bda6.

To quote:

    Our current approach of setting owner/mode in the Puppet way risks
    losing any extra permissions beyond owner/group/everyone, which is
    significant enough that it is better to simply not support this
    operation until we have a better solution. replace_file isn't used in
    any code which actually runs on Windows at the moment, so this is
    a benign change.

...except that it isn't so benign as all that: various other parts of the code
open-code the same method, poorly, and with various different bugs.  They will
all, without question, lose permissions in exactly the same way on Windows.

At least by routing them through a central path we can start to unify their
behaviour and get to a place where fixing one point in the code fixes all the
faulty permission transfers that we previously had.

We win an overall net reduction in bug count, and an increase in
predictability of bugs, from this change.
Replace Puppet::Util::FileLocking with atomic `replace_file`

The Puppet::Util::FileLocking module had substantial complexity in aide of
allowing careful, synchronized updating of individual files.  It was super
careful, and wanted to be absolutely sure that it didn't have concurrent reads
and writes going on, so it:

 * used per-file Ruby level locks to avoid multiple threads accessing the file
   at the same time.
 * used file-system level cooperative file locking on the target file to avoid
   multiple processes seeing concurrent updates.
 * used a temporary file to avoid rewriting the target file in place, so there
   was no "partial file read" window.
 * carefully used portable constructs to achieve locking rather than
   OS-specific kernel level exclusions.

Of course, some of those claims were documented but no longer valid - using a
temporary file *and* cooperative file locking don't play nice, and the
temporary file bit had dropped from the actual implementation forever ago.

It also did nothing to satisfy any of the other interesting conditions around
files: if multiple concurrent writers began, there were no assurances beyond
chance in the OS process scheduler which would win and replace the file.

In practice all that was really wanted were some very, very simple semantics:

Any process or thread can read the file at any time, and will get a complete
and correct version of the file.

Any process or thread can write at any time, provided that the read constraint
is satisfied.

It turns out that we recently built a method with these exact semantics,
`replace_file`, which will perform an atomic replacement using rename, but
only if the new file content is safely and completely stored on disk.

That satisfies all current semantics: any process can read at any time, and
will either get the complete previous version, or the complete new version,
depending on how it schedules around the (POSIX defined atomic) rename.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Fix terminus testing for Ruby 1.9

The order of evaluation of abstract classes (Class.new) changed in Ruby 1.9,
resulting in the hacks here that worked around the "must be bound to a
constant" rules in the indirector starting to fail.

Technically, though, we don't actually care: we can define a whole class stack
properly, binding everything the way the system expects, and then clean it up
afterwards.

Less stubbing, better tests, and clearer logic, all of which are pretty nice.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Handle YAML load errors from the new Psych YAML parser.

Earlier versions of Ruby used the syck library to parse YAML; during 1.9 this
moved to the Psych parser based on libyaml.  This brings YAML 1.1 compliance
to the library, and generally moves in a forward direction.

It also makes some radical changes to the failure modes, since Psych is much
more forgiving of some YAML constructs that would previously block the heck
up.

It also, in Ruby tradition, does absolutely nothing to abstract over the
implementation: Ruby can use either Psych or Syck and will raise different
exceptions on failure in both cases.

Worse, Psych exceptions are not derived from StandardError, so
Psych::SyntaxError will actually bypass almost any normal attempt to trap the
error, and will have wildly different details to the Syck errors.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Action definition requires at least one argument.

Ruby 1.9 allows us to enforce the calling convention of face actions at load
time, rather than failing with a runtime error.  This, in turn, means that we
should define actions correctly in our tests, if we expect them to work.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Hash references on string now raise errors, not return nil.

Under Ruby 1.8 the syntax `"foo"[:bar]` would return `nil`, while Ruby 1.9 is
strict: it enforces that only numeric index arguments are acceptable and
raises an error when that precept is violated.

This means that a whole pile of specs that used to pass strings where
structured objects were accepted now blow up - because you can't request
arbitrary nonsense against their array dereference interface.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Eliminate dead home-made assertion-at-dev-time-only code.

The assertion code hidden in here, designed to allow development time
assertions than vanish at runtime - in the classic "better to keep working on
corrupt data and failed assumptions than fail in front of the user" mode - was
not actually used anywhere in the product.

We may as well not carry this little legacy around any further than it current
has been, then.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
`instance_variables` changes return type between 1.8 and 1.9

The return type of `instance_variables` changes between Ruby 1.8 and 1.9
releases; it used to return an array of strings in the form "@foo", but
now returns an array of symbols in the form :@foo.

Nothing else in the stack cares which form you get - you can pass the
string or symbol to things like `instance_variable_set` and they will work
transparently.

Having the same form in all releases of Puppet is a win, though, so we
pick a unification and enforce than on all releases.  That way developers
who do set math on them (eg: for YAML rendering) don't have to handle the
distinction themselves.

In the sane tradition, we bring older releases into conformance with newer
releases, so we return symbols rather than strings, to be more like the
future versions of Ruby are.

We also carefully support reloading, by only wrapping when we don't
already have the original version of the method aliased away somewhere.

This includes a second monkey-patch, on the `resolv` library that ships with
Ruby 1.8.  This is the only bit of code in the core stack that made
assumptions about the return format of names from `instance_methods`.

Specifically, it assumed they were strings, and could be evaluated using
`instance_eval` to fetch out the value.  The fix is to replace that with a
call to `instance_variable_get` which, like it does in Ruby 1.9, avoids the
cost of eval parsing as well as being more robust overall.

I did audit the rest of the standard library on 1.8.5 and 1.8.7 to verify
this.  We only need worry if we start using the SOAP or Tk support.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Syck specific YAML parser tests can only run with the old engine.

Ruby 1.9 moved to using the Psych engine as the default YAML engine, replacing
the older Syck based YAML engine.  They did this without any regard to parse
tree compatibility, or abstraction over the underlying tool.

Consequently, the test that gets very intimate with the encoding format for
the parsed YAML tree is unable to function with the modern engine - it tries
to interact with a pile of methods that just don't match.

This commit disables that test unless the Syck parser is in use.  Really, the
entire test should be an external, abstracted acceptance level test.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Calling map on string now fails, rather than yielding the string.

In Ruby 1.8 `"foo".map` would yield "foo" once, while in Ruby 1.9 it is not
defined as a method.

Various parts of our testing code stub other methods and return strings where
arrays of strings are appropriate; this historically worked because of that
1.8 shim, and no longer does.

The same is true of `String#each`, which fails in the same way.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
`String#to_a` doesn't work in Ruby 1.9

In Ruby 1.8, counterintuitively, `"foo".to_a` would return `["foo"]` - an
array with a single entry, the string.  This went away in Ruby 1.9, likely
as a transition path to a much less confusing set of semantics.

Either way we should stop relying on that behaviour and get with the program.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>