Allow overriding and cleanup the needed directory only.

While there improve a bit logging.