Only do extent build if necessary