Avoid object creation/destruction when possible.

Object creation and destruction, even over a short time-frame, is expensive,
so where we can avoid it we should.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
type: memoize key_attributes for types.

The key_attributes have about a 0.05 percent miss rate on the cache, and the
value is static for the life of the type, so we can avoid substantial numbers
of object allocations.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
tagging: document why fixing this horror is hard.

The tagging support module has an "API" that includes other objects directly
interacting with the way it stores data, including at the edges where we
serialize to the wire format for transport.

So, we can't change it.  Even if it sucks - it allocates lots of objects, it
uses very inefficient deduplication mechanisms - we are kind of stuck,
right now.  Fun times.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
lexer: remove some unused stats collection

The lexer was instrumented to identify the number of regex vs string tokens
matched in the system - presumably as part of making sure it was testing in
the optimal order.

At this point we know the outcomes, so the collection can be removed.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
monkey_patch: cache Symbol to_proc result.

If we use it, our monkey-patch version of Symbol to_proc is worth caching the
resulting proc - the hit rate is through the roof, because we only really act
on a small set of symbols.

This reduces some extra object and memory allocation by reusing the same proc
every time we transform the same symbol.

That saves 99.84 percent of the memory allocated from this method - about four
percent of the total memory allocation in my test run of Puppet with real
manifests and behaviours.  (About 10MB out of 260MB originally; 26kb after.)

This applies to all versions - newer Rubies with a real implementation of
to_proc don't cache the result, so we do it for them.

Older versions *cough* 1.8.5 *cough* still get our hand-written implementation
of the proc, so that they work. It probably is equivalent to the real version,
but I would rather allow core changes to show.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
type: support implicit "identity" transformation

Almost every implementation of `title_patterns` on a type returned with a
lambda that performed an "identity" transformation - `lambda {|x| x }`.

That actually cost a notable amount of memory allocation, because it created a
new instance of the lambda on every invocation. That represented about three
percent of total memory allocation, or 8MB total.

This allows the `title_patterns` data to simply omit the lambda - if you don't
supply one, we don't invoke it to transform the result, just assign directly.

This allows everyone to keep using this the way they currently do, including
supplying an identity lambda if they want, but allows internal types and types
that don't override the default to be more memory efficient.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
FileCollection is a memory loss, not a memory win.

The file type used the FileCollection object to try and reduce memory
consumption by deduplicating path strings.

Unfortunately, this had the opposite effect in the real world, even when many
duplicate paths were present - it cost about 1.5MB extra memory in my tests.

It also costs non-trivial CPU time reconstituting the original path every time
we need to interact with that, and we do that a bunch.

So, just ditch the entire thing and move folks to just using paths directly.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Eliminate remaining uses of FileCollection.

As noted, the whole FileCollection concept was an overall loss - it used more
memory, more CPU, and added more complexity.

This eliminates the rest of the uses of the tool, and turns it into an empty
stub that avoids breaking the external API of things that included it, but
removes any unification that it tried to do.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
parser/files: use a cheaper test for "is this path absolute?"

The parser used `path == File.expand_path(path)` as a test for the path being
absolute - something that wasn't entirely accurate, and that also wasn't
terrible fast. It also generated substantial memory and object allocation
traffic in the VM.

Replacing that with our util absolute_path? tool is faster and more efficient.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
util: don't rebuild regexp every time we invoke absolute_file?

Previously, the `absolute_path?` tool rebuilt both regular expressions, POSIX
and Windows, every time it was invoked.  Since they are constant, promoting
them to a constant saves some time and trouble for the VM.

We can also tell it that you can interpolate the values once on Windows,
rather than every time through, which should improve performance and object
load further on that platform.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
zaml: don't sort object attributes for output

Sorting YAML attributes makes for marginally nicer human-friendly YAML output,
but ends up costing a non-trivial amount of computation to determine the
ordering of objects.

We can skip that with pretty much no adverse effects.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
provider: don't create useless proc instances.

When defining commands, the code created an empty proc in the 129 out of 130
cases that the caller didn't supply one, on my test system.  That was an awful
lot of uselessly defined code to avoid "unless block" in the target method.

Which we do, and avoid that load.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
whit: cache the `to_s` result.

This saves about 1.6MB of generated memory, something on the order of 10,000
object allocations and garbage collections, and a whole lot of waste in my
test system.

This runs at, like, O(50n), so while it is just linear it is still a huge
constant factor through the system.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
zaml: speed micro-optimization for Time output

This is a minor reduction in the time spent outputting times in the ZAML
encoder, by using a (30 percent) faster mechanism to trim the generated ms
value down.

There is a minor reduction in memory load, but not anything substantial.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
log_paths: memoize the entire value, not just part of it.

This function would memoize the upstream input, but would then recalculate the
final string value every time through.  That doesn't make much sense, so we
might as well memoize the entire value.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
file_serving: avoid rewriting paths if possible.

A trivial change - only rewrite from '//' to '/', rather than from '/+' -
means that we can reduce the garbage generation in this
function substantially.  Most paths now pass through unmodified.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
tagging: use a static regexp rather than rebuilding it

This is worth ~ 1 percent CPU time in catalog compilation in testing.

Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
Merge branch 'perf/3.x/reduce-puppet-memory-footprint' into 3.x

* perf/3.x/reduce-puppet-memory-footprint:
  tagging: use a static regexp rather than rebuilding it
  file_serving: avoid rewriting paths if possible.
  log_paths: memoize the entire value, not just part of it.
  zaml: speed micro-optimization for Time output
  whit: cache the `to_s` result.
  provider: don't create useless proc instances.
  zaml: don't sort object attributes for output
  util: don't rebuild regexp every time we invoke absolute_file?
  parser/files: use a cheaper test for "is this path absolute?"
  Eliminate remaining uses of FileCollection.
  FileCollection is a memory loss, not a memory win.
  type: support implicit "identity" transformation
  monkey_patch: cache Symbol to_proc result.
  lexer: remove some unused stats collection
  tagging: document why fixing this horror is hard.
  type: memoize key_attributes for types.
  Avoid object creation/destruction when possible.

Conflicts:
	lib/puppet/provider.rb