Add new restart handling sample from CiSE draft.

New version shows off using a subset of the communicator for the
metadata creation.  It also comments on/fixes a race condition
in this use case.  Bettter comments as well.