saltstack/salt — Issue #531: salt-master stops responding after a few hours
Author: archtaku
Created: 2012-01-18 05:51:09+00:00
State: open

My salt master stops working, without fail, after a period of a few hours. One of the salt-master threads goes zombie, and all commands fail with an empty dict as output.

This is happening with a source install from git, done with setup.py. The master and the minion are running on the same host, if that matters.

$ sudo salt '*' cmd.retcode 'ls -l'
{}

$ ps -ef | grep salt-master
root 7342 1 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
root 7343 7342 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
root 7350 7342 0 Jan17 ? 00:00:00 [salt-master]
root 7353 7342 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
root 7356 7342 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
root 7359 7342 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
root 7360 7342 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
root 7365 7342 0 Jan17 ? 00:00:00 /usr/bin/python26 /usr/bin/salt-master -d
ejohnson 8771 8458 0 00:31 pts/1 00:00:00 grep salt-master

The log for the salt master does not show any errors logged.

I'm not sure how to go about troubleshooting, but am open to suggestion.