opscode-cookbooks/chef-client — pull request #217 by davidstrauss (2014-11-20 08:21:39+00:00)
Title: Fix and modernize the systemd service configuration

The current systemd service configuration is broken. This is what happens on CentOS 7:

[root@samlcert ~]# systemctl status chef-client.service
chef-client.service - Chef Client daemon
Loaded: loaded (/etc/systemd/system/chef-client.service; enabled)
Active: deactivating (stop-sigterm) since Thu 2014-11-20 08:04:38 UTC; 6s ago
Process: 2527 ExecStart=/usr/bin/chef-client -d -c $CONFIG -P $PIDFILE -i $INTERVAL -s $SPLAY (code=exited, status=0/SUCCESS)
Main PID: 2529 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/chef-client.service
└─2532 /opt/chef/embedded/bin/ruby /usr/bin/chef-client -d -c /etc/chef/client.rb -P /var/run/chef/client.pid -i 1800 -s 300

Nov 20 08:04:37 samlcert systemd[1]: Starting Chef Client daemon...
Nov 20 08:04:38 samlcert chef-client[2527]: [2014-11-20T08:04:38+00:00] INFO: Daemonizing..
Nov 20 08:04:38 samlcert systemd[1]: Started Chef Client daemon.
Nov 20 08:04:38 samlcert chef-client[2527]: [2014-11-20T08:04:38+00:00] INFO: Forked, in 2532. Privileges: 0 0

About 60 seconds after it reaches that, systemd kills it because Chef's forking approach doesn't work with systemd's expectations. And then you get this:

[root@samlcert ~]# systemctl status chef-client.service
chef-client.service - Chef Client daemon
Loaded: loaded (/etc/systemd/system/chef-client.service; enabled)
Active: failed (Result: timeout) since Thu 2014-11-20 08:06:08 UTC; 8s ago
Main PID: 2529 (code=exited, status=0/SUCCESS)

Nov 20 08:04:37 samlcert systemd[1]: Starting Chef Client daemon...
Nov 20 08:04:38 samlcert chef-client[2527]: [2014-11-20T08:04:38+00:00] INFO: Daemonizing..
Nov 20 08:04:38 samlcert systemd[1]: Started Chef Client daemon.
Nov 20 08:04:38 samlcert chef-client[2527]: [2014-11-20T08:04:38+00:00] INFO: Forked, in 2532. Privileges: 0 0
Nov 20 08:06:08 samlcert systemd[1]: chef-client.service stopping timed out. Killing.
Nov 20 08:06:08 samlcert systemd[1]: Unit chef-client.service entered failed state.

It's much simpler to run `chef-client` in the foreground within systemd. The only downside is that, if anything has `After=chef-client.service`, it will run immediately after systemd executes the Chef client, not after the Chef client drops a PID file. I think that's a worthwhile trade-off to have a working service. "Reload" still operates properly.

I've also dropped the After=syslog.target. I'm not aware of any distribution that has required that in any maintained release in the last year, certainly not Fedora or CentOS.

After the change, `chef-client.service` is a happy one, including with `systemctl reload chef-client.service`:

[root@samlcert ~]# systemctl status chef-client.service
chef-client.service - Chef Client daemon
Loaded: loaded (/etc/systemd/system/chef-client.service; enabled)
Active: active (running) since Thu 2014-11-20 08:06:20 UTC; 3min 17s ago
Process: 2611 ExecReload=/bin/kill -HUP $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 2603 (chef-client)
CGroup: /system.slice/chef-client.service
└─2603 /opt/chef/embedded/bin/ruby /usr/bin/chef-client -c /etc/chef/client.rb -i 1800 -s 300

Nov 20 08:06:20 samlcert systemd[1]: Started Chef Client daemon.
Nov 20 08:06:39 samlcert systemd[1]: Reloading Chef Client daemon.
Nov 20 08:06:39 samlcert systemd[1]: Reloaded Chef Client daemon.
Nov 20 08:06:39 samlcert chef-client[2603]: [2014-11-20T08:06:39+00:00] INFO: SIGHUP received, reconfiguring