Avoid rebuilding everything due to submodule update.

The factoring isn't right yet, but it should work well enough
for now.