celery/celery — pull request #1749 by brodie (2013-12-18 19:42:30+00:00)
Title: events.State: always instantiate a Worker for offline nodes

This addresses a regression in Celery 3.1.7 (introduced in ef15f051e09c6b8dd225a8bf1786e5a9f821d1cd) where workers crash when they receive events of other workers going offline that they hadn't seen go online:

```
celery.worker Unrecoverable error: AttributeError("'NoneType' object has no attribute 'hostname'",)
Traceback (most recent call last):
File "celery/worker/__init__.py", line 206, in start
self.blueprint.start(self)
File "celery/bootsteps.py", line 123, in start
step.start(parent)
File "celery/bootsteps.py", line 373, in start
return self.obj.start()
File "celery/worker/consumer.py", line 270, in start
blueprint.start(self)
File "celery/bootsteps.py", line 123, in start
step.start(parent)
File "celery/worker/consumer.py", line 786, in start
c.loop(*c.loop_args())
File "celery/worker/loops.py", line 72, in asynloop
next(loop)
File "kombu/async/hub.py", line 333, in create_loop
cb(*cbargs)
File "kombu/transport/base.py", line 156, in on_readable
reader(loop)
File "kombu/transport/base.py", line 141, in _read
drain_events(timeout=0)
File "amqp/connection.py", line 305, in drain_events
return amqp_method(channel, args, content)
File "amqp/channel.py", line 1900, in _basic_deliver
fun(msg)
File "kombu/messaging.py", line 589, in _receive_callback
return on_m(message) if on_m else self.receive(decoded, message)
File "celery/worker/consumer.py", line 775, in on_message
obj, subject = self.update_state(event)
File "celery/events/state.py", line 471, in event
return self._event(event)
File "celery/events/state.py", line 531, in _event
on_node_leave(worker)
File "celery/worker/consumer.py", line 732, in on_node_leave
debug('%s left', worker.hostname)
AttributeError: 'NoneType' object has no attribute 'hostname'
```

In my particular case, this was happening quite frequently as I was restarting Celery workers across many machines.