briancavalier/avow — pull request #1 by wizardwerdna (2013-04-04 21:23:56+00:00)
Title: crashes Node 0.10.x because of changes to nextTick

Avow fails pretty spectacularly under Node 0.10.x when too many promises are returned in .then callbacks.

Prior to 0.10.x, process.nextTick deferrals got highest priority (performed before the event loop) unless process.maxNextTick deferrals were queued, in which case, the system would give the event loop a spin before proceeding. In 0.10.x, this was changed to ASSURE that all nextTicks would be completed before the event lop was run again. Accordingly, "recursive" calls to nextTick from nextTick-scheduled functions can now run up the stack and crash the system.

>FASTER PROCESS.NEXTTICK

>In v0.8 (and before), the process.nextTick() function scheduled its callback using a spinner on the event loop. This usually caused the callback to be fired before any other I/O. However, it was not guaranteed.

>As a result, a lot of programs (including some parts of Node's internals) began using process.nextTick as a "do later, but before any actual I/O is performed" interface. Since it usually works that way, it seemed fine.

>However, under load, it's possible for a server to have a lot of I/O scheduled, to the point where the nextTick gets preempted for something else. This led to some odd errors and race conditions, which could not be fixed without changing the semantics of nextTick.

>So, that's what we did. In v0.10, nextTick handlers are run right after each call from C++ into JavaScript. That means that, if your JavaScript code calls process.nextTick, then the callback will fire as soon as the code runs to completion, but before going back to the event loop. The race is over, and all is good.

>However, there are programs out in the wild that use recursive calls to process.nextTick to avoid pre-empting the I/O event loop for long-running jobs. In order to avoid breaking horribly right away, Node will now print a deprecation warning, and ask you to use setImmediate for these kinds of tasks instead.

Running brian's [promise-perf-tests](https://github.com/cujojs/promise-perf-tests) under 0.10.x was very disturbing as nearly every library (except for when.js and jQuery, by the way!) died horrible deaths for various reasons. Avow, alas, was also a casualty

In the first of the attached commits, I exported one of the culprit tests to demonstrate the problem in avow. It seems that the problem arises when a .then callback returns a promise, many times along, leaving a trail of callbacks scheduled with nextTick, and boom. The test can be run with "mocha test/torture.js" (or using the "binary" in node_modules/.bin).

One solution, I have found, is simply to insert a "setIntermediate" before the promise-in-a-promise part of the .then apply function. This schedules that .then to occur after all pending nextTicks, while still assuring callback order preservation. There must be a better way, but none comes to mind at the moment. (When.js avoids the problem by literally setting up its own internal scheduler -- but I don't see why the C++ queuing code wouldn't be faster, by introducing a setImmediate in the "inner loop"). The remaining code passes the torture test without failing on the Promise/A+ test suite.

Thanks for all the good work brian.