Currently the reconnect logic will only retry once if EM.reconnect throws an exception. EM.reconnect can throw an exception in the case of a failed DNS lookup for example.

It looks as though in eventmachine if an EM.add_periodic_timer throws an exception no further calls to that block of code will be made. Presumably it reschedules itself after executing the callback which doesn't get called when an exception goes off. Arguably thats a bug with eventmachine? maybe?

Here is a simple program to demonstrate that (You'll only get one exception, not one every 5 seconds):

require 'eventmachine'

def throw_exception
    raise "Issues and problems"
end

def puts_running
  puts "Running...."
end

EM.run {
  EM.add_periodic_timer(1) { puts_running }
  EM.add_periodic_timer(5) { throw_exception }

  EM.error_handler do |e|
    puts "Eventmachine problem, #{e}"
    puts ("#{e.backtrace.join("\n")}")
  end
}

------------

It looks as though EM.connect has two ways of handling errors. The first being an exception (e.g. bad dns), the second is via callbacks.

This change simply handles any exception created by EM.reconnect so further retries are attempted.

Also adding a test case for this issue which will simulate a DNS failure to trigger an exception.