CMake: avoid a reconfigure loop from r213091

Removing the native CMakeCache.txt causes the target to get re-run needlessly
on some systems. We'll want another solution for that part of the fix.