Merge pull request #504 from pooya/jc_util_fixes

Jc util fixes
doc: Add an image about the worker protocol.

This image is from Prashanth's talk in Erlang Factory.
Merge pull request #505 from pooya/doc

doc: Add an image about the worker protocol.
job_coordinator: Do not start a new stage if there is one started already.

In the make_stage_tasks, when we are creating the next stage, we normally are
sure that it has not been created.  However, in the concurrent model we cannot
make this assertion.  Therefore, we make this function more strict in this
commit.
job_coordinator: Make sure a stage has finished before sending an event.

In the old model, the do_stage_done function is called only when a stage
finishes and there is no need for double checks.  However, in the concurrent
model, this function might be called when some tasks are finished.  Disco has to
make sure this stage can be finished before actually finishing it.
job_coordinator: extract wakeup_waiters out of task_complete.

task_complete use to do two separate things which made it complicated.
job_coordinator: Run the split tasks of the next stages concurrently.

If the next stage is of type split, we can start it even without finishing the
current stage.
disco_web: avoid showing negative numbers for the number of waiting tasks.
Merge pull request #506 from pooya/concurrent_split

Concurrent split
disco_server: Send the worker pid to the job scheduler.

This pid can be used for communicating to the worker and sending the future
inputs to it.
worker: Increase the worker protocol timeout.

Disco might have to prepare the inputs for the tasks and it certianly needs more
than 10 minutes to do that.  For concurrent stages, for some type of groupings,
Disco needs all of the tasks of the previous stage to finish.

The current timeout (3 days) is just a large value.  If any worker failed with a
timeout, we have to increase this value.
Merge pull request #507 from pooya/workerpid

Workerpid
worker: Exclude the inputs which are received in previous requests.
worker: fix the input termination condition.

Disco is done sending inputs to the worker when it sends the worker
the "done" message with the input.  Otherwise, the worker should
keep requesting inputs.
Merge pull request #509 from pooya/more_input

More input
doc: remove stale documentation about external workers.
erlang: avoid creating an extra fun when calling spawn_link.

This commit replaces some of spawn_link(Fun) and spawn_link(Node, Fun) with
spawn_link(Module, Function, Args) and spawn_link(Node, Module, Function, Args)
respectively.  The latter forms are prefered because of their simplicity.
Merge pull request #511 from pooya/new_spawn

erlang: avoid creating an extra fun when calling spawn_link.
disco_worker: Send the pid of the slave worker instead of the worker process on master.
job_coordinator: use the task_id in the task_started message.