Hope I am missing something here ...

After involing cleanup, use of rddA would result in errors - coupled with lazy execution, users would actually can end up cleaning up rdd's which have not yet been 'used'.
We should defer exposing this api until we have more clarity on this.

In particular, cleanup should ensure that all pending jobs which require the rdd should have finished.
Contrived example:
--
rdd3 = rdd1.join(rdd2)
rdd1.count()
rdd1.cleanup()
...
rdd3.count()
--

would cause issues for rdd3 - this is, arguably, bad code from spark dev point of view - but unlike with unpersist, where we can recover with performance panalty, with cleanup we will fail. From a (naive ?) users point of view, rdd1 has already been "used" by the time it is getting cleaned up.
Best option would be if we can throw an exception when cleanup is invoked in cases like this; and same exception being thrown if rdd is subsequently used (for any op) after a cleanup : this latter might be more an involved change though.