Code review comment: I updated the test code once more. Essentially, it will show the stacktrace of a generic error-case-specific `KeeperException`. For the `testShutdownWithFailureDueToExistingChildNodes` generates the following stacktrace: ``` java.util.concurrent.CompletionException: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter  	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375) 	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947) 	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounterITCase.testShutdownWithFailureDueToExistingChildNodes(ZooKeeperCheckpointIDCounterITCase.java:114) 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [...] 	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) 	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221) 	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54) Caused by: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter 	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:132) 	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.handleDeletionOfCounterPath(ZooKeeperCheckpointIDCounter.java:167) 	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.lambda$shutdown$0(ZooKeeperCheckpointIDCounter.java:131) 	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:926) 	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:683) 	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.DeleteBuilderImpl$2.processResult(DeleteBuilderImpl.java:207) 	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:675) 	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) ```  There's a log message printed on INFO beforehand in [ZooKeeperCheckpointIDCounter:L113](https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java#L113)

Code before review:
public CompletableFuture<Void> shutdown(JobStatus jobStatus) {
synchronized (startStopLock) {
if (isStarted) {
LOG.info("Shutting down.");
try {
sharedCount.close();
} catch (IOException e) {
return FutureUtils.completedExceptionally(e);
}
client.getConnectionStateListenable().removeListener(connectionStateListener);
if (jobStatus.isGloballyTerminalState()) {
LOG.info("Removing {} from ZooKeeper", counterPath);
final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
try {
client.delete()
.inBackground(
(curatorFramework, curatorEvent) -> {
Preconditions.checkArgument(
curatorEvent.getType()
== CuratorEventType.DELETE,
"An unexpected CuratorEvent was monitored: "
+ curatorEvent.getType());
Preconditions.checkArgument(
counterPath.equals(curatorEvent.getPath()),
"An unexpected path was selected for deletion: "
+ curatorEvent.getPath());
final KeeperException.Code eventCode =
KeeperException.Code.get(
curatorEvent.getResultCode());
if (Sets.immutableEnumSet(
KeeperException.Code.OK,
KeeperException.Code.NONODE)
.contains(eventCode)) {
deletionFuture.complete(null);
} else {
deletionFuture.completeExceptionally(
KeeperException.create(
KeeperException.Code.get(
curatorEvent
.getResultCode())));
}
})
.forPath(counterPath);
} catch (Exception e) {
Preconditions.checkState(
!deletionFuture.isDone(),
"The background task was triggered even though the curator call failed.");
deletionFuture.completeExceptionally(e);
}
return deletionFuture;
}
isStarted = false;
}
}
return FutureUtils.completedVoidFuture();
}

Line under review: .getResultCode())));

Code after addressing the review:
public CompletableFuture<Void> shutdown(JobStatus jobStatus) {
synchronized (startStopLock) {
if (isStarted) {
LOG.info("Shutting down.");
try {
sharedCount.close();
} catch (IOException e) {
return FutureUtils.completedExceptionally(e);
}
client.getConnectionStateListenable().removeListener(connectionStateListener);
if (jobStatus.isGloballyTerminalState()) {
LOG.info("Removing {} from ZooKeeper", counterPath);
try {
final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
client.delete()
.inBackground(
(curatorFramework, curatorEvent) ->
handleDeletionOfCounterPath(
curatorEvent, deletionFuture))
.forPath(counterPath);
return deletionFuture;
} catch (Exception e) {
return FutureUtils.completedExceptionally(e);
}
}
isStarted = false;
}
}
return FutureUtils.completedVoidFuture();
}