saltstack/salt-cloud — pull request #402 by whodatninja (2013-03-03 08:32:25+00:00)
Title: Fixed wait_for_ssh to prevent invalid socket state

My attempts to provision EC2 VMs were failing at the SSH step; meanwhile I had no trouble connecting to them directly using parameters identical to those I was giving salt-cloud.

After adding debug output to wait_for_ssh, I saw the following:

[DEBUG ] Attempting to connect to host on port 22
[ERROR ] Error connecting to SSH: [Errno 61] Connection refused
[DEBUG ] Retrying SSH connection (try 1)
[ERROR ] Error connecting to SSH: [Errno 22] Invalid argument
[DEBUG ] Retrying SSH connection (try 2)
[ERROR ] Error connecting to SSH: [Errno 22] Invalid argument
[DEBUG ] Retrying SSH connection (try 3)
[ERROR ] Error connecting to SSH: [Errno 22] Invalid argument
...
[DEBUG ] Retrying SSH connection (try 350)
[ERROR ] Error connecting to SSH: [Errno 22] Invalid argument

I'm no expert on python's socket module but obvious hypothesis was after first (entirely normal) error, socket enters an invalid state and -- because sock is initialized outside of the while loop -- is never properly reset. So I just moved `sock =` inside the try block. And that fixed it.

Also, I think it's a good idea to leave in these log.debug statements as I suspect SSH is a common source of baffling issues.