Close the test database before removing it, part 3

This should really fix #2 for good.