Change:
// First we check for scheduler messages, these are higher // priority than regular tasks. <del> let (sched, stask, did_work) = </del> <ins> let (mut sched, mut stask, mut did_work) = </ins> self.interpret_message_queue(stask, DontTryTooHard); <del> if did_work { return stask.put_with_sched(sched); } // This helper will use a randomized work-stealing algorithm // to find work. let (sched, stask, did_work) = sched.do_work(stask); if did_work { return stask.put_with_sched(sched); </del> <ins> // After processing a message, we consider doing some more work on the // event loop. The "keep going" condition changes after the first // iteration becase we don't want to spin here infinitely. // // Once we start doing work we can keep doing work so long as the // iteration does something. Note that we don't want to starve the // message queue here, so each iteration when we're done working we // check the message queue regardless of whether we did work or not. let mut keep_going = !did_work || !sched.event_loop.has_active_io(); while keep_going { let (a, b, c) = match sched.do_work(stask) { (sched, task, false) => { sched.interpret_message_queue(task, GiveItYourBest) } (sched, task, true) => { let (sched, task, _) = sched.interpret_message_queue(task, GiveItYourBest); (sched, task, true) } }; sched = a; stask = b; did_work = c; // We only keep going if we managed to do something productive and // also don't have any active I/O. If we didn't do anything, we // should consider going to sleep, and if we have active I/O we need // to poll for completion. keep_going = did_work && !sched.event_loop.has_active_io(); </ins> } <del> // Now, before sleeping we need to find out if there really // were any messages. Give it your best! let (mut sched, stask, did_work) = sched.interpret_message_queue(stask, GiveItYourBest); </del> <ins> // If we ever did some work, then we shouldn't put our scheduler // entirely to sleep just yet. Leave the idle callback active and fall // back to epoll() to see what's going on. </ins> if did_work { return stask.put_with_sched(sched); }

Discussion:
In the current design the scheduler takes one scheduling action, registers a callback to wake up in the runschedonce function, and then blocks on the event loop. This "block" is implemented as just finishing the execution associated with the runschedonce callback. As there is a callback registered it immediately wakes up and repeats the process. This should be unnecessary. The drop back to the event loop is a very long process relative to a task context swap, so this is likely very bad for performance. A better solution would be to keep the scheduler active until it runs out of stuff to do, and then block on the event loop. To do this we should have a "live" flag for the scheduler. If the flag is set the scheduler is guarenteed to check for messages and likely work before going to sleep. While active the scheduler algorithm becomes: 1) Call run on the event loop with the NOWAIT flag set. This will make it so that if there are no events, the function returns immediately instead of blocking. 2) Find a task to run, and run it. Repeat a small number of times to get better task throughput. 3) If the live flag is false, call a blocking run. 4) If no work has been done this round, set the live flag to false. 5) Goto (1) The libuv "uvrun" function has three modes. UVRUNDEFAULT is what we have been using, a blocking "do stuff until the program is done" use of the event loop. UVRUNNOWAIT polls for new events and then does not block if there are no events ready. This is the style that we want for the non-blocking run case. UVRUNONCE is how we would perform blocking runs. It will block until it handles a single event, and then return how many pending events there are. To wake up the scheduler while it is "sleeping", the old idle-callback is used. Except now the function provided simply modifies the live flag to "awake", and the scheduler loop logic takes over. In any case where an idle callback would be registered and the live flag is set the callback can be skipped.
This builds on somewhat as it also changes how idle callbacks work in the scheduler.

As far as I can tell the structure you suggest isn't about eliminating 'blocking' per se - if there are events to be run then the loop doesn't block - but is a proposal to improve performance by using polling instead of events. From the description I believe that the performance win you expect to get here is from not using the idle callback. I'm skeptical that calling the idle callback is a significant amount of the work that the event loop is doing though. The work imposed by an idle callback on is the following (from loop-watcher.c): That is, traversing an element of a linked list and calling a function pointer. This structure has a downside that it would impose more requirements on our generic event loop, and that it is more complicated. I'm not inclined to make this change without evidence that the current callback-based structure is a problem.
no longer relevant to the standard libraries ()