(PUP-914) Use resource ref when writing dot files

Use the `ref` attribute of a resource instead of `to_s`.

Internally, Puppet uses the `ref` attribute to identify and locate resources
within a catalog. This patch makes use of `ref` to identify nodes in graphviz
output as some Types, notably Whit, override `to_s` and either destroy or
obscure the identity of resource nodes.
(PUP-3505) Fix problem with passing explicit undef res parameter

With the future parser in effect and a resource attribute is undef the
logic that checks for missing arguments did not report the parameter.
This in turn led to that the logic that either fails the operation, or
assigns the default value to completely skip that parameter/argument.
Subsequently a reference of that parameter ended up picking the same
variable from an outer context.

The fix is to equate a nil value (4x) with an :undef value (for 3x) when
computing the missing arguments.

A test is added that tests the functionality.
(PUP-3500) Avoid cacheing environment values when handling setting hooks

Environment in parse_config is being used to ensure that settings in the
current run_mode's environment are checked for hooks, and to handle
hooks that are simply on_write.  In this latter case, the code looks up
the value of the setting from all available elements of the setting's
search_path including the current environment layer.

But if directory environments are in use, their environment settings
cannot be looked up at this early point in settings initialization (we
have no environment loaders), and the consequence is that we end up with
a ChainedValues instance cached in the Settings @values[:env][:runmode]
that does not have a section for :env.  This then blocks resolution of
the real environment settings at a later date.

The fix in this patch is to construct a temporary ChainedValues instance
for the lookup rather than invoking value(), thus avoiding the cacheing.

Since none of the documented environment settings has a hook, we were
very tempted to yank out the environment handling entirely, but this
would leave us with yet more odd behavior if a hook was added to an
environment setting in the 3.7.x series for some unknowable reason, and
it was then set in a legacy environment block.

* the question of how to handle Setting hooks for directory environment
settings is open in PUP-3507

Paired-with: Britt Gresham <britt@puppetlabs.com>
Merge pull request #3228 from jpartlow/issue/stable/pup-3500-avoid-cacheing-chained-values-when-handling-setting-hooks

(PUP-3500) Avoid cacheing environment values when handling setting hooks
(PUP-3370) Remove serialization support from indirector request

Support for serializing/deserializing indirector requests was added per
Redmine issue #7080, but was never fully fleshed out and is a rather
sketchy idea in the first place. This commit removes all serialization
logic from Puppet::Indirector::Request.
(PUP-3277) Use classifier when testing enc response

The current test sets up an enc instance for foss, but when running
against pe, it fails because the pe node_terminus configuration is in
[master] not [main].  We could adjust that, but then we
are not testing against a real PE environment, so instead, use the
classifier.
Merge branch 'issue/master/pup-3277-enc-non-existent-directory-environment-test-uses-classifier-in-pe'

* issue/master/pup-3277-enc-non-existent-directory-environment-test-uses-classifier-in-pe:
  (PUP-3277) Use classifier when testing enc response
(PUP-3300) Remove extend self from Checksums

This commit modifies Puppet::Util::Checksums to not extend self anymore
to move to using module_function.
(PUP-3289) Remove deprecated catalog_format and dynamicfacts settings

This commit is part of the Puppet 4 code removal effort.
(PUP-2609) Make copying permissions on Win an error

Copying permissions on Windows are problematic, so it isn't done. Change
the deprecation warning about source_permissions => use/use_when_creating
to an error.
Merge remote-tracking branch 'upstream/pr/3230'

* upstream/pr/3230:
  (PUP-3370) Remove serialization support from indirector request
(PUP-3130) Remove hidden _timestamp fact

Puppet generates a timestamp when a Puppet::Node::Facts object is
created. During PSON serialization (from agent to master), the timestamp
is serialized separately from the hash of fact values.

Previously, when the master deserialized the facts from JSON, it would
convert the timestamp to a hidden fact named `_timestamp`, inline with
the normal facts. As a result, there was a `_timestamp` variable
available in top-scope.

The `_timestamp` fact was added in 5aa4440b circa 2007. It was
originally called `_puppet_timestamp`, and then renamed to `_timestamp`
in a6fe7005.

This commit modifies the Fact object so that it no longer adds a
`_timestamp` fact during deserialization. Note the timestamp is still
available, via the Facts#timestamp method, it's just not a fact.

This commit also removes the magic behavior of stripping facts with
leading underscores. This improves performance because we no longer
duplicate the values hash, or apply a regex to every fact key.

It also preserves behavior of the compiler, added in fb5f859cf4, which
updates the facts timestamp prior to saving them.
Merge pull request #3229 from hlindberg/PUP-3505_given-undef-not-considered-missing

(PUP-3505) Fix problem with passing explicit undef res parameter
(PUP-2609) Update Metadata default permission

Update Metadata's default permission model to ignore source permissions,
and remove unreachable code. Update spec tests for new behavior.
Merge pull request #3232 from whopper/pup-3289/removal/cat_format-dynfacts

(PUP-3289) Remove deprecated catalog_format and dynamicfacts settings
Merge pull request #3231 from demophoon/refactor/master/pup-3300-checksum-refactor

(PUP-3300) Remove extend self from Checksums
(PUP-2289) Fix host type newline validation

This commit fix the first bug reported on PUP-2289.
This is done by implementing validation of newlines
on ip, hostname, comments and host_aliases, making
sure we don't have any \n or \r.

The bug and fix is on host resource type.

Below is the problem description:

If the ip address or hostname or host_aliases
use double quoting and have \n, Puppet Will:

1 - Write the wrong entry on /etc/hosts with the linebreak.
This leads to a corrupted /etc/hosts, with some lines
that could not be parsed by the host resource provider
parsed.

This leads to second bug on the second puppet run,
where it will have error after failing to parse the file,
but wipes out the /etc/hosts file, writing only the information
on that file that is managed by puppet.

How to reproduce it:
1- backup /etc/hosts
cp /etc/hosts /etc/hosts.orig
2 - Create the script to reproduce the bug 01
cat <<EOF > /tmp/hostsbug.pp
host {'hostsbug': ip => "10.10.12.1\n" }
EOF
The same apply if you have newline on hostname, hostaliases or comment.
3 - Run the script the first time to reproduce bug 01.
puppet apply /tmp/hostsbug.pp
4 - Check /etc/hosts
cat /etc/hosts

Leonardo Rodrigues de Mello

PUP-2289 Fix bug on host resource provider

The host resource provider parsed file wipes
all entries of etc/hosts file if it gets one
entry that could not be parsed.

This commits fix this problem, considering that
incomplete lines are :text_lines. This is the same
solution that was used for mount resource provider.

I  had checked that etc/hosts ignores invalid entries
at least on linux, tested on rhel 5, 6, 7 and debian 5,6,7 and
fedora 20.

It would be great if someone could test it on other platforms.

Thanks
Leonardo Rodrigues  de Mello

Fixes as required on the pull request
Merge pull request #3235 from kylog/lmello-fix_pup-2289

(PUP-2289) Fix host type newline validation
(maint) Remove some shims for compat with 0.25.x
(maint) Report rubocop failures correctly

Previously rubocop would run and find defects, but the exit
code was not reported. The Jenkins job was relying on the exit
code, so would never catch failures.