Remove the tricky buffer handling now that we have Buffered{Reader,Writer}

Signed-off-by: David Scott <dave.scott@citrix.com>