client: add an option for non pnfs read/write

Target: master
Acked-by: Gerd Behrmann