Remove cache and benchmark files for now.