Use ByteBufferAccessor indirection in fewer cases