Merge pull request #930 from daniel-pittman/perf/3.x/2892-json-terminus-for-catalog-cache

#2892: json terminus for catalog cache
(maint) Add Puppet::Parser::Scope.new_for_test_harness

Without this patch there is no clear path into the internals of Puppet
for third parties who intended to exercise Puppet parser functions.

This is a problem because many 3rd parties, such as the stdlib module
and hiera-puppet extend Puppet using parser functions.  These modules
create instances of Puppet::Parser::Scope objects in their test harness
to invoke the parser functions they're testing.

The specific problem this patch means to address is the dependency of
Puppet modules on the specific method signature of
Puppet::Parser::Scope#initialize  This patch addresses the problem my
adding a new method, Puppet::Parser::Scope.new_for_test_harness which
accepts a single argument, the node name, and returns a scope instance
bound to a compiler instance and a node instance.

The common spec helper should be augmented to invoke this method.  All
Puppet modules that exercise parser functions should then decouple
themselves from Puppet itself and instead depend upon the common spec
helper behavior.

Reviewed-by: Patrick Carlisle <patrick@puppetlabs.com>
Merge branch 'fix/3.x/harness_the_scope' into 3.x

* fix/3.x/harness_the_scope:
  (maint) Add Puppet::Parser::Scope.new_for_test_harness

Reviewed-by: Patrick Carlisle <patrick@puppetlabs.com>
Performance: remove some occurences of type.to_s.downcase

Those are either DRYed or plain unneeded.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
Pull out initialization of resource class? and stage?

Some slight modification to Brice's patch
Performance: memoize resource#resource_type

Because calling Puppet::Type.type forces a full and slow autoloading
session with a negative result when the given resource type is a
defined type, we therefore cache the result (there's no chance this
value would change during the compilation).
This saves about 20% of compilation time on my setup.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
Performance: TypeCollection should remember nonexistent types before autoloading

When the TypeCollection doesn't find a class/definition it tries to
autoload it in each namespace. This requires calling TypeLoader#import
for this qualified name, which is a time consuming operation.

When loading definitions/classes that can't be found by the autoloader
(ie legacy manifests for instance),  the autoloader will still try to
load them in the autoloaded module path, even though those types have
possibly already been loaded (ie directly imported).  Searching for
those definitions/classes then becomes really expensive.

This problem is frequent for people (like me) that migrated manifests
from the 0.24/0.25 era without refactoring all their modules to strictly
follow the autoloading mechanism.

Now, we negatively cache when we couldn't load a type with the
autoloader, so that the next attempt we won't spend our time trying
again.

This saves about 30% of compilation time on my setup

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
Performance: template wrapper calling info is slow

On every variable lookup of templates we're trying to gather where
originated this operation in the template. This is time consuming
and needed only when the variable can't be found.
So we now lazily fetch this information only when we actually need
the line information.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
Merge branch 'performance-3.x-brice' into 3.x

* performance-3.x-brice:
  Performance: template wrapper calling info is slow
  Performance: TypeCollection should remember nonexistent types before autoloading
  Performance: memoize resource#resource_type
  Pull out initialization of resource class? and stage?
  Performance: remove some occurences of type.to_s.downcase
Merge branch 'perf/3.x/reduce-puppet-memory-footprint' into 3.x

* perf/3.x/reduce-puppet-memory-footprint:
  tagging: use a static regexp rather than rebuilding it
  file_serving: avoid rewriting paths if possible.
  log_paths: memoize the entire value, not just part of it.
  zaml: speed micro-optimization for Time output
  whit: cache the `to_s` result.
  provider: don't create useless proc instances.
  zaml: don't sort object attributes for output
  util: don't rebuild regexp every time we invoke absolute_file?
  parser/files: use a cheaper test for "is this path absolute?"
  Eliminate remaining uses of FileCollection.
  FileCollection is a memory loss, not a memory win.
  type: support implicit "identity" transformation
  monkey_patch: cache Symbol to_proc result.
  lexer: remove some unused stats collection
  tagging: document why fixing this horror is hard.
  type: memoize key_attributes for types.
  Avoid object creation/destruction when possible.

Conflicts:
	lib/puppet/provider.rb
Merge remote-tracking branch 'dpittman/perf/3.x/only-one-symbol-to-zaml' into 3.x

Conflicts:
	lib/puppet/util/monkey_patches.rb
Merge remote-tracking branch 'dpittman/perf/3.x/eliminate-type-attribute-aliases' into 3.x

* dpittman/perf/3.x/eliminate-type-attribute-aliases:
  Eliminate Puppet type attribute aliases.

Conflicts:
	lib/puppet/type.rb
Disable ActiveSupport::Dependencies hooks.

The ActiveSupport::Dependencies code is loaded along with Rails, and hooks
into a bunch of the system.  This consumes a surprisingly large amount of CPU
time watching for constants and generally acting to support unloading code.

This is related to the code that automatically requires libraries when the
constant is first accessed.

We don't - can't, since Rails is a soft dependencies - depend on that feature,
or anything related to it, we have no benefit from paying that horrible price.

Dropping this is actually a significant drop in CPU consumption - some 8
percent of the CPU cost in my testing is attributed directly to this feature.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>