Update provider case e2e tests to be more robust again