Repository: tensorflow
Issue: //tensorflow/python/client:session_partial_run_test is flaky
Labels: stat:awaiting tensorflower,type:bug,type:build/install,subtype: ubuntu/linux,TF 2.12

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

git HEAD

### Custom Code

No

### OS Platform and Distribution

Ubuntu 20.04

### Mobile device

n/a

### Python version

3.9.16

### Bazel version

5.3.0

### GCC/Compiler version

10.2.1

### CUDA/cuDNN version

n/a

### GPU model and memory

n/a

### Current Behaviour?

Unit test reports as FLAKY or FAILED.

See https://source.cloud.google.com/results/invocations/dea422ff-7e14-4fc1-b324-0129ecd7ffbc/log or https://github.com/tensorflow/tensorflow/actions/runs/4731924097/jobs/8397430880#step:5:23224

### Standalone code to reproduce the issue

```shell
docker exec tf bazel --bazelrc=/usertools/cpu.bazelrc test --config=rbe --config=pycpp --config=build_event_export
```


### Relevant log output

```shell
======================================================================
ERROR: testPartialRunMissingPlaceholderFeedExceptionDist (__main__.PartialRunTest)
PartialRunTest.testPartialRunMissingPlaceholderFeedExceptionDist
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1379, in _do_call
    return fn(*args)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1369, in _prun_fn
    return self._call_tf_sessionprun(handle, feed_dict, fetch_list)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1460, in _call_tf_sessionprun
    return tf_session.TF_SessionPRun_wrapper(self._session, handle, feed_dict,
tensorflow.python.framework.errors_impl.InternalError: From /job:localhost/replica:0/task:0:
ValidateDevices called before initialization.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py", line 1629, in decorated
    return f(self, *args, **kwargs)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session_partial_run_test.py", line 269, in testPartialRunMissingPlaceholderFeedExceptionDist
    self.RunTestPartialRunMissingPlaceholderFeedException(
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session_partial_run_test.py", line 119, in RunTestPartialRunMissingPlaceholderFeedException
    sess.partial_run(handle, fetches[0])
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1026, in partial_run
    return self._run(handle, fetches, feed_dict, None, None)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1192, in _run
    results = self._do_run(handle, final_targets, final_fetches,
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1375, in _do_run
    return self._do_call(_prun_fn, handle, feeds, fetches)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/client/session_partial_run_test.runfiles/org_tensorflow/tensorflow/python/client/session.py", line 1398, in _do_call
    raise type(e)(node_def, op, message)  # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InternalError: Graph execution error:

From /job:localhost/replica:0/task:0:
ValidateDevices called before initialization.

----------------------------------------------------------------------
Ran 25 tests in 2.625s

FAILED (errors=1, skipped=1)
================================================================================
```
</details>