Project for the `CompilerSettingsTest` tests is now created programmatically

This commit will be squashed.