cfengine/core — pull request #244 by estenberg (2012-11-24 22:04:13+00:00)
Title: agent timeout

The motivation for this feature is that some systems have been seen to have hundreds of cf-agent/cf-twin processes spawned by cf-execd - to the point where they have to be rebooted.
The root causes may vary, and it is not necessarily CFEngine's "fault" (e.g. disk partition full, command never terminates, system call hangs, etc.), but the effect should be mitigated.

This feature introduces a timeout on each run of cf-agent. If cf-execd sees that the timeout is reached, it will kill the agent. The agent (or a command executed by it) can reset the timer by outputting something to STDOUT.

### Current issues
* Windows port - ProcessSignalTerminate() is not defined, could use GracefulTerminate() on Windows.
* Children of cf-agent (e.g. processes in commands: promises) may not be killed correctly (needs confirmation). We may need to kill the entire process tree.

### Testing

Save the following to inputs/promises.cf, and run cf-execd -v.

body common control
{
bundlesequence => { "test" };
}

bundle agent test
{
commands:
"/bin/sleep 2000";
}

body executor control
{
schedule => { "any" };
exec_command => "/var/cfengine/bin/cf-agent";
agent_expireafter => "1";
}

After running it for a while, you can check:

$ ps -ef|grep cf-agent
user 12539 6147 0 13:55 pts/0 00:00:00 sh -c /var/cfengine/bin/cf-agent -Dfrom_cfexecd
user 12540 12539 0 13:55 ? 00:00:00 /var/cfengine/bin/cf-agent -Dfrom_cfexecd

$ ps -ef|grep sleep
user 6155 1 0 13:30 ? 00:00:00 /bin/sleep 2000
user 6177 1 0 13:32 ? 00:00:00 /bin/sleep 2000
user 6183 1 0 13:33 ? 00:00:00 /bin/sleep 2000
user 6207 1 0 13:35 ? 00:00:00 /bin/sleep 2000
user 6311 1 0 13:36 ? 00:00:00 /bin/sleep 2000
user 6374 1 0 13:37 ? 00:00:00 /bin/sleep 2000
user 6391 1 0 13:38 ? 00:00:00 /bin/sleep 2000
user 6428 1 0 13:39 ? 00:00:00 /bin/sleep 2000
user 6434 1 0 13:40 ? 00:00:00 /bin/sleep 2000
user 8856 1 0 13:42 ? 00:00:00 /bin/sleep 2000
user 12391 1 0 13:44 ? 00:00:00 /bin/sleep 2000
user 12429 1 0 13:45 ? 00:00:00 /bin/sleep 2000
user 12436 1 0 13:46 ? 00:00:00 /bin/sleep 2000
user 12447 1 0 13:48 ? 00:00:00 /bin/sleep 2000
user 12454 1 0 13:49 ? 00:00:00 /bin/sleep 2000
user 12471 1 0 13:50 ? 00:00:00 /bin/sleep 2000
user 12499 1 0 13:52 ? 00:00:00 /bin/sleep 2000
user 12523 1 0 13:53 ? 00:00:00 /bin/sleep 2000
user 12530 1 0 13:54 ? 00:00:00 /bin/sleep 2000
user 12542 12540 0 13:56 ? 00:00:00 /bin/sleep 2000

So just one cf-agent is running, but a lot of sleep processes are still running. Note that init has taken ownership over them, but they still seem to be running.