SPARK-1830 Deploy failover, Make Persistence engine and LeaderAgent Pluggable

I agree; only the `persist()`, `unpersist()`, `read()`, and `close()`methods are designed for extensibility; everything else should be `final`.