puppetlabs/puppet — pull request #1158 by jeffmccune (2012-09-17 21:47:32+00:00)
Title: (#2888) Fix race condition with puppetdlockfile

Without this patch applied there is a race condition where by two or
more concurrent puppet agent processes will eventually result in a
deadlock state where all processes consider catalog runs to be
administratively disabled.

This is a problem because the configuration catalog stops being applied
periodically. The system will not recover unless there is manual
intervention by the end user.

The problem is caused by the settings catalog. The settings catalog
will contain a file resource for the puppetdlockfile if the file exists.
When two processes are running, the file will exist when it is created
by the other process. When the file resource is in the settings
catalog, it will be created as a zero length file, or truncated to a
zero length file if it already exists.

Here is the state transition for two puppet agent --test processes, A
and B.

A: (Start of the catalog run, after settings catalog is applied)
lock()
File.exist? => false
File.open()
catalog.run
...

B: (Managing settings catalog, before user catalog run)
catalog.add_resource(Puppet::Type::File)
Puppet::Util::Settings#to_catalog
...
resource synchronize
Puppet::Type::File#write(:content)

A:
unlock()
unlink()

B:
# apply
File.open("puppetdlock", "wb") { ... } # Truncates the file!A

A: run
locked? => true
mine? => false

B: run
locked? => true
mine? => false

A and B are deadlocked.

This patch fixes the problem by marking the puppetdlockfile settings as
a `:type => :setting`. This change prevents the settings catalog from
containing a file resource for the puppetdlockfile setting.

Paired-with: Josh Cooper