Change the semantics of requesting executors

Instead of requesting executors incrementally, the interface
between the driver and the AM is now setting the number of
pending executors instead of the number of additional executors
to request. This commit also adds the relevant documentation
to explain this change, along with other missing documentation
(esp. in the Yarn allocation code) that clarifies the intent
of parts of the existing code.

TODO: The existing mechanism is still subject to the following
race condition: the driver may get more executors than requested
if it makes a second request AFTER the cluster manager has already
addressed the first request, but BEFORE the executors in the first
request have registered. This will be addressed in the next commit.
Fix race condition that causes over-allocation of executors

In addition to the number of pending executors to request, we
also tell the cluster manager (or AM in our case) the number of
executors we currently have. This means the AM can account for
any divergence in the number of executors that have registered
according to the driver and the same number according to the AM.
Otherwise, from the number of pending executors to request alone
it is impossible for the AM to tell whether a prior request has
already been satisfied.

We may want to consider simplifying this further by specifying
the total number of executors we want, since we just add the
pending and running executors now anyway.
Simplify the request interface by asking for a total

This means we do not distinguish between the number of executors
pending vs running. We only ever used to add up the two anyway,
so functionally it makes no difference. The interface exposed to
the scheduler backend is also simpler to reason about.