Ensure we cleanup some files so session dirs can be rolled up

cmr=v1.8.2:reviewer=jsquyres