Correctly handle read-only direct ByteBuffer when wrap them