In general, these "Info" objects are not intended to be serialized to workers, which is why it lives in the master package. It would be nice to maintain this abstraction if possible, to keep what we're sending between Master and Worker simple.

A simpler way to do this might be a "best-effort" method, where the Master keeps a local retryCount in its DriverInfo, which it decrements every time it launches the driver. Additionally, it could pass its current value of this retryCount to the worker via the LaunchDriver message, which becomes a local variable within DriverRunner (no need for DriverInfo at all there).

If the DriverRunner fails due to too many retries, the Master will not relaunch the Executor, otherwise it essentially assumes it failed once. This may cause the driver to actually be restarted more than "spark.driver.maxretrynum" or whatever, but I don't think the actual retry count matters as much as the fact that any such limit exists.

This shouldn't be a very big change to your patch, and mainly a simplification. What do you think?