Prevent memory copying in _fit