CLC-2289 Second followup: Be more discriminating in what we delete

https://jira.media.berkeley.edu/jira/browse/CLC-2289