Replace home-grown network IO stack with tornado for katcp server

Doing so lets us use the tried and tested goodness of tornado. Also gives us some async coroutine goodies for 'free'. This is a big change, and I would like several people to review it.

Since most of our codebase is not async-aware, the DeviceServer can be used in three modes, depending on the options passed to the set_concurrency_options() method.

1. Fully async mode. All code must be aware of the fact that all IO calls need to be made from the ioloop thread, and request handlers must **never** block. Currently won't work because of SamplingReactor working in a separate thread.
2. Thread safe IO, but request handlers must still be non-blocking. Performance overhead due to thread safety management.
3. Full compatibility mode where the request handlers are all run in a single thread, meaning that the message-handling loops of all connections are blocked by a single slow handler. Messages can still be sent when the message-handling loops are blocked, meaning sensor-updates continue. This mirrors the previous katcp concurrency semantics, but without the CPU-busy-loop send nonsense on slow clients. Unline the previous katcp implementation, sending to a slow client won't block all message-handling loops. This mode should not require any changes to users of the katcp.DeviceServer class, and is, for now, the default.

As a side-benefit, the unused ExceptHookThread class was removed.

Upcoming Work
=============

1. Make the DeviceServer  handle_message function support coroutine handlers, or anything returning a future. Should not be too hard.
2. Implement SamplingReactor to use ioloop and not own separate event loop. Needed to support Fully async mode. Might also clean up sampling to support a generic callback that takes the sensor reading rather than the KATCP formatted message, in a backwards compatible way.
3. Make client classes use tornado. Should be able to re-use the read-loop from the server here.

Looks good!
OK to commit IMHO.


On Fri, Aug 15, 2014 at 10:23 AM, Neilen Marais <notifications@github.com>
wrote:

> In katcp/server.py:
>
> > +        set_ioloop(), start(), stop(), join(), send_message(), mass_send_message(), and
> > +        _async() versions of the send messages from another thread. Except for the
> > +        _async() versions, all calls must be made from  the ioloop thread
> > +        """
> > +
> > +        self._device = device
> > +        self._bindaddr = (host, port)
> > +        self._tb_limit = tb_limit
> > +        self._logger = logger
> > +        self._parser = MessageParser()
> > +        # Indicate that server is running and ready to accept connections
> > +        self._running = threading.Event()
> > +        # Indicate that we are stopped, i.e. join() can return
> > +        self._stopped = threading.Event()
> > +        self.ioloop = None
> > +        "The Tornado IOloop to use, set by self.set_ioloop()"
>
> The one for self.ioloop? That is an attribute docstring, it's in our
> coding guidlines :)
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/ska-sa/katcp-python/pull/9/files#r16283876>.
>