Well, because all these statements never get executed as far as I understand :)
The block in the with statement only gets executed until the point where the exception occurs.

I found that by accident because one of these checks is testing for an error containing "assert" while the test is for erroring out when the project dir is named "values". You can enter whatever string you want in these checks, they never happen.

I thought about that the basic idea to test the error in this detail is not too bad, but as I couldn't come up with a way to do it easily, I thought again and wondered if it's really likely that we get a false positive test here because the config parser errors out with a different reason, and come to the conclusion it's not very likely and so it's ok to just remove the non executed tests and not think about too much how to replace them with something that really does what they were intended to do.