saltstack/salt — pull request #4456 by fabiant7t (2013-04-10 14:12:27+00:00)
Title: Fixed #4411 (salt-minion on Ubuntu: 100% CPU after Master rejected the public key)

# Solution
In case a Salt Minion got rejected by the Salt Master, it exits with the return code 0 (instead of 42). Upstart got configured not to respawn Salt Minion if it was exiting with return code 0.

# Test
I was testing the fix on a fresh Ubuntu 12.0.4.1 server on Amazon AWS:
1) Install git (sudo apt-get install git)
2) Bootstrap salt from my git repo (curl -L http://bootstrap.saltstack.org | sed 's/github.com\/saltstack/github.com\/fabiant7t/g' | sudo sh -s -- git develop)
3) Set the salt master (set master: [YOURSALTMASTER] in /etc/salt/minion)
4) Restart Salt Minion (sudo service salt-minion restart)
5) On Salt Master: Reject Salt Minion's public key
6) Restart Salt Minion (sudo service salt-minion restart)
7) Make sure salt-minion exited and did not get respawned (ps -C salt-minion -f)
```
ubuntu@ip-10-226-118-73:~$ ps -C salt-minion -f
UID PID PPID C STIME TTY TIME CMD
```
8) Make sure there is just one current log message saying that Salt Minion got rejected (sudo tail -n 5 /var/log/salt/minion)
```
ubuntu@ip-10-226-118-73:~$ sudo tail -n 5 /var/log/salt/minion
2013-04-10 13:49:19,853 [salt.crypt ][ERROR ] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2013-04-10 13:49:30,203 [salt.crypt ][ERROR ] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
2013-04-10 13:49:37,711 [salt.crypt ][CRITICAL] The Salt Master has rejected this minion's public key!
To repair this issue, delete the public key for this minion on the Salt Master and restart this minion.
Or restart the Salt Master in open mode to clean out the keys. The Salt Minion will now exit.
```
9) Make sure there is just one current Upstart log message saying that Salt Minon got rejected (sudo tail -n 5 /var/log/upstart/salt-minion.log)
```
[ERROR ] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
[ERROR ] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate
[CRITICAL] The Salt Master has rejected this minion's public key!
To repair this issue, delete the public key for this minion on the Salt Master and restart this minion.
Or restart the Salt Master in open mode to clean out the keys. The Salt Minion will now exit.
```
10) Make sure the normal exit status code is set in the Upstart config for Salt Minion (sudo cat /etc/init/salt-minion.conf | grep normal)
```
ubuntu@ip-10-226-118-73:~$ sudo cat /etc/init/salt-minion.conf | grep normal
normal exit 0
```