Only create ByteBuf if needed