Add tests to run in the different configurations, and check the configurations work correctly.

This also fixes a couple of issues with not waiting for the server to shutdown before running the next test.