cmullaparthi/ibrowse — pull request #122 by benjaminplee (2014-11-17 16:03:45+00:00)
Title: Status pipeline algorithm changes?

My team has been using iBrowse for some time now in one of our products without issue, thank you, but some recent scale testing found that the current find_best_connection logic was problematic with our internal load balancers between systems. After reviewing the current code we have fielded (slightly behind current master e18ea912b62b0f4a48d0f0edc8bb2f542863a88c), I see our problem is with consistent attempts at giving the "first" connection the next request, even if others have fewer outstanding in their pipeline or round-robin. I was about to begin on a patch to address this when I saw your unmerged new_pipeline branch which appears to address this same issues, along with some other items.

I am curious as to the status of this branch and what if anything you see as remaining before it can be called complete. As we have an immediate need for a modified algorithm, I am happy to help close out this feature branch or work to separate out the specific algorithm changes from the others. Please let me know what I can do to help drive this to completion.

I am also wondering about your choice to use a timestamp as part of the key-tuple for connection lookup in ets. Are you looking for order of creation to be preserved if two connections have the same speculative size? In our usage, we have a max sessions size of ~512 for traffic between our servers (network load balancer inbetween) and are seeing a significant skew of load (e.g. the first server in a 3 wide cluster gets 10-20% more load than the second, which gets 10-20% more load than the first). I believe I have traced this back to the repeated offering of work to the first connection before offering to others. I would like to find a way to more evenly distribute requests across connections with the same pipeline depth in order to ensure the long running connections don't skew overtime.