hudl/Mjolnir — pull request #31 by robhruska (2014-09-29 03:26:15+00:00)
Title: Isolate (limit concurrency) using TaskSchedulers instead of thread pools

Getting this out there for some early review. Quite unfinished, but will pick it back up in a couple weeks.

The problem: Mjolnir's been using thread pools to isolate calls by limiting concurrency. This isn't a bad approach, but has its drawbacks:
- Thread pools are kind of expensive (See #19)
- The implementation we use (SmartThreadPool) is terrible about preserving context
- SmartThreadPool hasn't accepted my PR to support bounded queues, which means we have to deliver the DLL manually (See #16).
- SmartThreadPool doesn't have great support for `Task`s, which resulted in some wrapping and `.Get()` blocking calls.

I originally wanted to replace our thread pools with a queue+semaphore, but having researched a bit more, I think custom TaskSchedulers would be way better. In fact, there's [an MSDN example](http://msdn.microsoft.com/en-us/library/ee789351(v=vs.110).aspx) of a concurrency-limiting TaskScheduler that I just yoinked into this PR (and modified) to prove the concept.

Since the `TaskScheduler`'s going to be really good with `Task`s, I'm planning on just removing most of the thread pool related bits and some of the abstractions (e.g. `IWorkItem`). We can add them back in if we need to later.

Still a lot to do here. Tests are passing, but I need some specific tests for the scheduler and some more integration-like tests to check that Exceptions get thrown and caught from the right place when we reject calls or when the isolated operation throws an exception.