Remove empty directories after tests.

I believe we don't need tests for this :)