pika/pika — Issue #100: pika hangs ioloop with blocking connect()
Author: brandon-rhodes
Created: 2011-11-01 18:47:18+00:00
State: open

Instead of letting the application register a callback to be invoked when a socket has either succeeded in connecting or has failed to connect, Pika freezes the entire IO loop by doing a synchronous connect(). From `base_connection.py` the offending code is:

self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
self.socket.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
self.socket.connect((host, port))
self.socket.setblocking(0)

The problem here is that `connect()` is called before `setblocking(0)` which freezes the application until the connection attempt has been resolved either way. On my Ubuntu 10.10 machine, for example, this means that an attempt to connect to an AMQP instance that happens to be down at the moment results in an IO loop frozen for more than 3 minutes — during which time the IO loop cannot move messages between any of the AMQP servers that have in the meantime remained up!

I did see an email to the Pika mailing list earlier this year about some kind of a rewrite following negative comments you got at PyCon regarding async, but I would like to push back against those negative comments and argue that the world very badly needs a good Python AMQP library that works with a Tornado IOLoop, and I think that Pika is great at this. I would love to see the last synchronous bits of Pika fixed and see the current design go 1.0, because I think that it is a very good design. For very simple problems, a synchronous library like `amqplib` is fine; but as soon as things start getting busy, I always find myself having to rewrite things as an async IO loop — and the Tornado pattern with ØMQ (for internal app communication) and Pika (to talk to queues out on open networks) are a great combination!