Defer API prefetch until it's required

In normal operation, after we get the first level dependencies, we'll
make several prefetch calls before we need one of those results. Waiting
until then means we can make fewer HTTP requests.
Merge pull request #871 from matthewd/defer_fetch

Defer API prefetch until it's required