Move some build task work away from the configure phase.

Reviewed at https://reviews.apache.org/r/20062/