  We study online learning for optimal allocation when the resource to be
allocated is time. Examples of possible applications include a driver filling a
day with rides, a landlord renting an estate, etc. Following our initial
motivation, a driver receives ride proposals sequentially according to a
Poisson process and can either accept or reject a proposed ride. If she accepts
the proposal, she is busy for the duration of the ride and obtains a reward
that depends on the ride duration. If she rejects it, she remains on hold until
a new ride proposal arrives. We study the regret incurred by the driver first
when she knows her reward function but does not know the distribution of the
ride duration, and then when she does not know her reward function, either.
Faster rates are finally obtained by adding structural assumptions on the
distribution of rides or on the reward function. This natural setting bears
similarities with contextual (one-armed) bandits, but with the crucial
difference that the normalized reward associated to a context depends on the
whole distribution of contexts.
