nskinkel/oppy — pull request #67 by nskinkel (2014-11-19 22:59:29+00:00)
Title: Pipeline refactoring

Hi all.

This pull request is a restructuring of the way we deal with connections, circuits, and streams. It allows multiple connections, circuits, and streams to be used simultaneously.

A connection pool has been implemented so that connections are now created when a circuit requests them and automatically torn down when they are no longer in use. If multiple circuits use the same entry node, the will share the TLS connection.

Circuits are now requested from circuit manager, which maintains a pool of open circuits. We can set some constraints on the type of default circuits we want to maintain. Everytime an incoming stream needs a circuit, circuit manager will first check to see if it already has an open circuit that the stream can use. If so, that stream will be assigned to the open circuit. If not, a new circuit will be built and added to the circuit manager's circuit pool.

When a circuit is no longer being used by any streams, it reports that it is currently unused to the circuit manager, and the circuit manager decides whether or not that circuit should be destroyed.

Streams are created on demand from socks. When a new stream is created, it will request a circuit. When the stream is closed, it reports it's closing to it's associated circuit.

Additionally, code has been refactored to be more readable and easier to debug. Use of deferreds is minimized, and connection/circuit/stream assignments are made explicit.

While a framework exists to handle protocol control tasks (flow control, circuits torn down remotely) and errors (receiving unexpected cells, connections being unexpectedly torn down), most of these tasks are currently unhandled. When an error happens, exception is raised, or a protocol control message comes in that we can't yet handle, it's currently just logged. Thankfully, since we can now spin up lots of simultaneous circuits, a user won't really notice. If a circuit fails the next stream will just choose a new one.

These are all cases that we were not previously handling either.

I will give a much more thorough overview of what's going on during tomorrow's meeting, and hopefully we can have a group code review.

Basically, this branch adds no new functionality in terms of different cells or ability to handle errors/protocol control messages. It simply allows us to share connections, pool circuits, and multiplex streams across circuits.

If you have time, please take a few minutes and read through the code before our meeting tomorrow.