Delete files that would not be a part of distribution