LPS-43812 Test that it works as expected with a non-existing context path

Here are the tests, as promised cc/ @csierra