Repository: PaddlePaddle/Paddle (language: python)

Issue:
[WIP][PHI] Fix 0 size Tensor initilalized judgement
<!-- TemplateReference: https://github.com/PaddlePaddle/Paddle/wiki/PULL-REQUEST-TEMPLATE--REFERENCE -->
<!-- Demo: https://github.com/PaddlePaddle/Paddle/pull/24810 -->

### PR Category
<!-- One of [ User Experience | Execute Infrastructure | Operator Mechanism | CINN | Custom Device | Performance Optimization | Distributed Strategy | Parameter Server | Communication Library | Auto Parallel | Inference | Environment Adaptation ] -->

Execute Infrastructure

### PR Types
<!-- One of [ New features | Bug fixes | Improvements | Performance | BC Breaking | Deprecations | Docs | Devs | Not User Facing | Security | Deprecations | Others ] -->

Bug fixes

### Description
<!-- Describe what you’ve done -->

0 size Tensor 初始化判断不应该包含 `ptr()` 的判断，只需要 holder 存在即可，0 size 即便没有 Allocation 也应该算是初始化完成

目前因为 `__deepcopy__` 实现里判断了 0 size Tensor `initialized`，而 0 size 永远为 False，导致了 holder 没有被 copy，进一步造成了后续动转静中设置动态图 OutMeta 时访问 place 挂掉

CI/CE 应该会暴露一些问题，会在暴露后确认方案是否合适，并进行修复

cc @HydrogenSulfate 

PCard-66972
[WIP][PHI] Fix 0 size Tensor initilalized judgement
<!-- TemplateReference: https://github.com/PaddlePaddle/Paddle/wiki/PULL-REQUEST-TEMPLATE--REFERENCE -->
<!-- Demo: https://github.com/PaddlePaddle/Paddle/pull/24810 -->

### PR Category
<!-- One of [ User Experience | Execute Infrastructure | Operator Mechanism | CINN | Custom Device | Performance Optimization | Distributed Strategy | Parameter Server | Communication Library | Auto Parallel | Inference | Environment Adaptation ] -->

Execute Infrastructure

### PR Types
<!-- One of [ New features | Bug fixes | Improvements | Performance | BC Breaking | Deprecations | Docs | Devs | Not User Facing | Security | Deprecations | Others ] -->

Bug fixes

### Description
<!-- Describe what you’ve done -->

0 size Tensor 初始化判断不应该包含 `ptr()` 的判断，只需要 holder 存在即可，0 size 即便没有 Allocation 也应该算是初始化完成

目前因为 `__deepcopy__` 实现里判断了 0 size Tensor `initialized`，而 0 size 永远为 False，导致了 holder 没有被 copy，进一步造成了后续动转静中设置动态图 OutMeta 时访问 place 挂掉

CI/CE 应该会暴露一些问题，会在暴露后确认方案是否合适，并进行修复

cc @HydrogenSulfate 

PCard-66972


Discussion / hints:
你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考[Paddle-CI手册](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/paddle_ci_manual_cn.html)。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See [Paddle CI Manual](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/paddle_ci_manual_cn.html) for details.
你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考[Paddle-CI手册](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/paddle_ci_manual_cn.html)。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See [Paddle CI Manual](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/paddle_ci_manual_cn.html) for details.

Fix patch:
diff --git a/paddle/fluid/eager/eager_tensor.h b/paddle/fluid/eager/eager_tensor.h
index 1ab743ff53cd3..c6d4d2942f610 100644
--- a/paddle/fluid/eager/eager_tensor.h
+++ b/paddle/fluid/eager/eager_tensor.h
@@ -130,6 +130,8 @@ class VariableCompatTensor
 
   bool valid() const override { return IsInitialized(); }
 
+  bool has_allocation() const override { return IsInitialized(); }
+
   bool initialized() const override { return IsInitialized(); }
 
   void* AllocateFrom(phi::Allocator* allocator UNUSED,
diff --git a/paddle/fluid/pir/dialect/operator/ir/ir_selected_rows.h b/paddle/fluid/pir/dialect/operator/ir/ir_selected_rows.h
index dd41a7467a78d..f1e06713eba13 100644
--- a/paddle/fluid/pir/dialect/operator/ir/ir_selected_rows.h
+++ b/paddle/fluid/pir/dialect/operator/ir/ir_selected_rows.h
@@ -74,6 +74,11 @@ class IrSelectedRows
 
   bool valid() const noexcept override { return true; }
 
+  bool has_allocation() const override {
+    PADDLE_THROW(::common::errors::Unavailable(
+        "`has_allocation` is only available at runtime"));
+  }
+
   bool initialized() const override { return true; }
 
   void* AllocateFrom(phi::Allocator* allocator,
diff --git a/paddle/fluid/pir/dialect/operator/ir/ir_sparse_tensor.h b/paddle/fluid/pir/dialect/operator/ir/ir_sparse_tensor.h
index 7844644db3de6..7f5be23294fea 100644
--- a/paddle/fluid/pir/dialect/operator/ir/ir_sparse_tensor.h
+++ b/paddle/fluid/pir/dialect/operator/ir/ir_sparse_tensor.h
@@ -75,6 +75,11 @@ class IrSparseCooTensor
 
   bool valid() const noexcept override { return true; }
 
+  bool has_allocation() const override {
+    PADDLE_THROW(::common::errors::Unavailable(
+        "`has_allocation` is only available at runtime"));
+  }
+
   bool initialized() const override { return true; }
 
   void* AllocateFrom(phi::Allocator* allocator,
@@ -151,6 +156,8 @@ class IrSparseCsrTensor
 
   bool valid() const noexcept override { return true; }
 
+  bool has_allocation() const override { return true; }
+
   bool initialized() const override { return true; }
 
   void* AllocateFrom(phi::Allocator* allocator,
diff --git a/paddle/fluid/pir/dialect/operator/ir/ir_tensor.h b/paddle/fluid/pir/dialect/operator/ir/ir_tensor.h
index 7d58bb63e5586..4b1f0e5e8037f 100644
--- a/paddle/fluid/pir/dialect/operator/ir/ir_tensor.h
+++ b/paddle/fluid/pir/dialect/operator/ir/ir_tensor.h
@@ -73,6 +73,11 @@ class IrTensor : public phi::TensorBase,
 
   bool valid() const noexcept override { return true; }
 
+  bool has_allocation() const override {
+    PADDLE_THROW(::common::errors::Unavailable(
+        "`has_allocation` is only available at runtime"));
+  }
+
   bool initialized() const override { return true; }
 
   void* AllocateFrom(phi::Allocator* allocator,
diff --git a/paddle/fluid/pybind/eager_method.cc b/paddle/fluid/pybind/eager_method.cc
index 7ad3780cdb3d5..193ea5a217b7a 100644
--- a/paddle/fluid/pybind/eager_method.cc
+++ b/paddle/fluid/pybind/eager_method.cc
@@ -693,11 +693,11 @@ static PyObject* tensor_method_copy_(TensorObject* self,
     egr::EagerUtils::autograd_meta(&(self->tensor))
         ->SetPersistable(
             egr::EagerUtils::autograd_meta(&(src_tensor))->Persistable());
-    if (src_tensor.initialized()) {
+    if (src_tensor.has_allocation()) {
       self->tensor.copy_(src_tensor, src_tensor.place(), blocking);
     }
   } else {
-    if (src_tensor.initialized()) {
+    if (src_tensor.has_allocation()) {
       eager_gil_scoped_release guard;
       self->tensor.copy_(src_tensor, self->tensor.place(), blocking);
     }
diff --git a/paddle/phi/api/include/tensor.h b/paddle/phi/api/include/tensor.h
index a4ce550f9858c..773e7135dcc15 100644
--- a/paddle/phi/api/include/tensor.h
+++ b/paddle/phi/api/include/tensor.h
@@ -498,6 +498,13 @@ class PADDLE_API Tensor final {
    */
   bool defined() const;
 
+  /**
+   * @brief Determine whether Tensor has allocation
+   *
+   * @return bool
+   */
+  bool has_allocation() const;
+
   /**
    * @brief Determine whether Tensor is initialized.
    *
diff --git a/paddle/phi/api/lib/tensor.cc b/paddle/phi/api/lib/tensor.cc
index b7e66762cb970..ed22289a2329f 100644
--- a/paddle/phi/api/lib/tensor.cc
+++ b/paddle/phi/api/lib/tensor.cc
@@ -411,6 +411,10 @@ void Tensor::set_name(const std::string &name) { name_ = name; }
 
 bool Tensor::defined() const { return impl_ != nullptr; }
 
+bool Tensor::has_allocation() const {
+  return defined() && impl_->has_allocation();
+}
+
 bool Tensor::initialized() const { return defined() && impl_->initialized(); }
 
 bool Tensor::is_initialized() const {
diff --git a/paddle/phi/api/lib/tensor_method.cc b/paddle/phi/api/lib/tensor_method.cc
index 9a64ca6d3eae2..bdf121e757478 100644
--- a/paddle/phi/api/lib/tensor_method.cc
+++ b/paddle/phi/api/lib/tensor_method.cc
@@ -87,7 +87,7 @@ Tensor::copy_to<phi::dtype::float16>(const Place &target_place) const;
 void Tensor::copy_(const Tensor &src,
                    const phi::Place &target_place,
                    bool blocking) {
-  if (!src.initialized()) {
+  if (!src.has_allocation()) {
     VLOG(8) << "Src is empty, skip copy";
     return;
   }
diff --git a/paddle/phi/core/dense_tensor.h b/paddle/phi/core/dense_tensor.h
index c2d804199d2c7..46d61efbc1d9a 100644
--- a/paddle/phi/core/dense_tensor.h
+++ b/paddle/phi/core/dense_tensor.h
@@ -128,6 +128,10 @@ class TEST_API DenseTensor : public TensorBase,
   /// return Whether the allocation is allocated.
   bool initialized() const override { return holder_ && holder_->ptr(); }
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const override { return holder_ != nullptr; }
+
   /// \brief Allocate memory with requested size from allocator.
   /// \return The mutable data pointer value of type T.
   void* AllocateFrom(Allocator* allocator,
diff --git a/paddle/phi/core/distributed/auto_parallel/dist_tensor.cc b/paddle/phi/core/distributed/auto_parallel/dist_tensor.cc
index 10f3d8b3245ac..14b048e3c5512 100644
--- a/paddle/phi/core/distributed/auto_parallel/dist_tensor.cc
+++ b/paddle/phi/core/distributed/auto_parallel/dist_tensor.cc
@@ -273,6 +273,10 @@ bool DistTensor::valid() const {
 
 bool DistTensor::defined() const { return value_->holder_ != nullptr; }
 
+bool DistTensor::has_allocation() const {
+  return value_->holder_ != nullptr && (value_->holder_->ptr() || numel() != 0);
+}
+
 bool DistTensor::initialized() const {
   return value_->holder_ != nullptr && value_->holder_->ptr();
 }
diff --git a/paddle/phi/core/distributed/auto_parallel/dist_tensor.h b/paddle/phi/core/distributed/auto_parallel/dist_tensor.h
index 93efdb76dc8a4..8c1e23ed9fbc3 100644
--- a/paddle/phi/core/distributed/auto_parallel/dist_tensor.h
+++ b/paddle/phi/core/distributed/auto_parallel/dist_tensor.h
@@ -147,6 +147,10 @@ class DistTensor final
   /// \return The global dims of the dist tensor.
   const DDim& local_dims() const;
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const override;
+
   /// \brief Returns the global number of elements contained in tensor.
   /// \return The number of elements contained in tensor.
   int64_t numel() const override;
diff --git a/paddle/phi/core/extended_tensor.cc b/paddle/phi/core/extended_tensor.cc
index a57759b818488..0f1770d5661a9 100644
--- a/paddle/phi/core/extended_tensor.cc
+++ b/paddle/phi/core/extended_tensor.cc
@@ -47,6 +47,11 @@ bool ExtendedTensor::valid() const {
       "ExtendedTensor does not support `valid` method."));
 }
 
+bool ExtendedTensor::has_allocation() const {
+  PADDLE_THROW(common::errors::Unavailable(
+      "ExtendedTensor does not support `has_allocation` method."));
+}
+
 bool ExtendedTensor::initialized() const {
   PADDLE_THROW(common::errors::Unavailable(
       "ExtendedTensor does not support `initialized` method."));
diff --git a/paddle/phi/core/extended_tensor.h b/paddle/phi/core/extended_tensor.h
index 73cae43c0b54c..685972b94e535 100644
--- a/paddle/phi/core/extended_tensor.h
+++ b/paddle/phi/core/extended_tensor.h
@@ -47,6 +47,8 @@ class TEST_API ExtendedTensor : public TensorBase {
 
   bool valid() const override;
 
+  bool has_allocation() const override;
+
   bool initialized() const override;
 
   void* AllocateFrom(Allocator* allocator,
diff --git a/paddle/phi/core/selected_rows.h b/paddle/phi/core/selected_rows.h
index 145f7e7d3b2e4..8240846fde8ac 100644
--- a/paddle/phi/core/selected_rows.h
+++ b/paddle/phi/core/selected_rows.h
@@ -160,6 +160,10 @@ class SelectedRows : public TensorBase,
   /// \return Whether the metadata is valid.
   bool valid() const noexcept override { return impl_->valid(); }
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const override { return impl_->has_allocation(); }
+
   /// \brief Test whether the storage is allocated.
   /// return Whether the storage is allocated.
   bool initialized() const override { return impl_->initialized(); }
diff --git a/paddle/phi/core/selected_rows_impl.h b/paddle/phi/core/selected_rows_impl.h
index bec14d3604136..e676cdfe4a014 100644
--- a/paddle/phi/core/selected_rows_impl.h
+++ b/paddle/phi/core/selected_rows_impl.h
@@ -179,6 +179,10 @@ class SelectedRowsImpl {
   /// \return Whether the metadata is valid.
   bool valid() const noexcept { return value_->valid(); }
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const { return value_->has_allocation(); }
+
   /// \brief Test whether the storage is allocated.
   /// return Whether the storage is allocated.
   bool initialized() const { return value_->initialized(); }
diff --git a/paddle/phi/core/sparse_coo_tensor.h b/paddle/phi/core/sparse_coo_tensor.h
index c59d09f653513..db5b11a2ce7ab 100644
--- a/paddle/phi/core/sparse_coo_tensor.h
+++ b/paddle/phi/core/sparse_coo_tensor.h
@@ -126,6 +126,10 @@ class SparseCooTensor : public TensorBase,
   /// \return Whether the non_zero_elements_ metadata is valid.
   bool valid() const noexcept override { return non_zero_elements_.valid(); }
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const override { return values().has_allocation(); }
+
   /// \brief Test whether the non_zero_elements_ storage is allocated.
   /// In special cases, when nnz=0, non_zero_elements_ will not need to be
   /// initialized, but it is necessary to return true here, otherwise the
diff --git a/paddle/phi/core/sparse_csr_tensor.h b/paddle/phi/core/sparse_csr_tensor.h
index b746694475ade..ca9feca201374 100644
--- a/paddle/phi/core/sparse_csr_tensor.h
+++ b/paddle/phi/core/sparse_csr_tensor.h
@@ -130,6 +130,10 @@ class SparseCsrTensor : public TensorBase,
   /// \return Whether the non_zero_elements_ metadata is valid.
   bool valid() const noexcept override { return non_zero_elements_.valid(); }
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const override { return values().has_allocation(); }
+
   /// \brief Test whether the non_zero_elements_ storage is allocated.
   /// In special cases, when nnz=0, non_zero_elements_ will not need to be
   /// initialized, but it is necessary to return true here, otherwise the
diff --git a/paddle/phi/core/string_tensor.h b/paddle/phi/core/string_tensor.h
index 4fecd83d557cf..b2faac25ca9a8 100644
--- a/paddle/phi/core/string_tensor.h
+++ b/paddle/phi/core/string_tensor.h
@@ -97,6 +97,10 @@ class StringTensor : public TensorBase,
   /// \return Whether the metadata is valid.
   bool valid() const noexcept override { return meta_.valid(); }
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  bool has_allocation() const override { return holder_ != nullptr; }
+
   /// \brief Test whether the storage is allocated.
   /// return Whether the storage is allocated.
   bool initialized() const override { return holder_ && holder_->ptr(); }
diff --git a/paddle/phi/core/tensor_array.cc b/paddle/phi/core/tensor_array.cc
index cb89f01cdc065..6bdfe132db1c2 100644
--- a/paddle/phi/core/tensor_array.cc
+++ b/paddle/phi/core/tensor_array.cc
@@ -21,6 +21,21 @@ TensorArray::TensorArray(const std::vector<DenseTensor>& vec) {
   tensors_ = vec;
 }
 
+/// \brief Test whether the holder is created.
+/// \return Whether the holder is created.
+bool TensorArray::has_allocation() const {
+  if (tensors_.empty()) {
+    return false;
+  }
+
+  for (auto const& tensor : tensors_) {
+    if (!tensor.has_allocation()) {
+      return false;
+    }
+  }
+  return true;
+}
+
 /// \brief Test whether the tensor's storage in TensorArray is allocated.
 /// return Whether all tensors in TensorArray is allocated.
 bool TensorArray::initialized() const {
diff --git a/paddle/phi/core/tensor_array.h b/paddle/phi/core/tensor_array.h
index 3c17217bf0d6d..1c402f640461a 100644
--- a/paddle/phi/core/tensor_array.h
+++ b/paddle/phi/core/tensor_array.h
@@ -78,6 +78,10 @@ class TensorArray : public TensorBase,
   /// \brief This overridden function is not used in TensorArray.
   TEST_API bool valid() const override;
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  TEST_API bool has_allocation() const override;
+
   /// \brief Test whether the tensor's storage in TensorArray is allocated.
   /// return Whether all tensors in TensorArray is allocated.
   TEST_API bool initialized() const override;
diff --git a/paddle/phi/core/tensor_base.h b/paddle/phi/core/tensor_base.h
index 71045b12132bc..e60c1818b12fa 100644
--- a/paddle/phi/core/tensor_base.h
+++ b/paddle/phi/core/tensor_base.h
@@ -52,6 +52,10 @@ class TensorBase {
   /// \return Whether the metadata is valid.
   virtual bool valid() const = 0;
 
+  /// \brief Test whether the holder is created.
+  /// \return Whether the holder is created.
+  virtual bool has_allocation() const = 0;
+
   /// \brief Test whether the storage is allocated.
   /// \return Whether the storage is allocated.
   virtual bool initialized() const = 0;


Test patch:
diff --git a/test/legacy_test/test_eager_tensor.py b/test/legacy_test/test_eager_tensor.py
index c279172d6c0a5..9b9130b7917b8 100644
--- a/test/legacy_test/test_eager_tensor.py
+++ b/test/legacy_test/test_eager_tensor.py
@@ -542,6 +542,17 @@ def test_deep_copy(self):
                 np.array(selected_rows.get_tensor()),
             )
 
+    def test_deep_copy_0size_tensor(self):
+        x = paddle.to_tensor(np.array([]))
+        x_copy = copy.deepcopy(x)
+        self.assertEqual(x_copy.stop_gradient, x.stop_gradient)
+        self.assertEqual(x_copy.persistable, x.persistable)
+        self.assertEqual(x_copy.type, x.type)
+        self.assertEqual(x_copy.dtype, x.dtype)
+        self.assertEqual(x_copy.shape, x.shape)
+        self.assertEqual(str(x_copy.place), str(x.place))
+        np.testing.assert_array_equal(x.numpy(), x_copy.numpy())
+
     # test some patched methods
     def test_set_value(self):
         var = paddle.to_tensor(self.array)