Improve handling of timed builds

This fixes several issues with the way we were handling builds with timing information