Repository: uxlfoundation/oneTBB (C++)

Issue:
Assertion failure in task_handle destruction due to object size mismatch in polymorphic cleanup：m_private_counter >= 0 failed (located in the destroy function, line in file: 141)
# Summary
The task_handle_task base class assumes it knows the size of derived objects during destruction, causing memory pool counter mismatches and assertion failures. When task_handle objects are destroyed through their unique_ptr custom deleter, destruction happens through base class pointer, leading to incorrect object size calculations and race conditions.

# Version
oneTBB 2022.0.0

# Environment
Hardware: Apple Silicon (M4 Pro) 
OS: macOS (all versions)
Compiler: Apple Clang (debug builds)
Build Type: Debug 

# Observed Behavior

[TBB_DESTROY] final_private_counter=-1 (will assert if < 0)
[TBB_DESTROY] *** ASSERTION WILL FAIL ***
[TBB_DESTROY] This indicates a race condition in USD's TBB usage
**Assertion m_private_counter >= 0 failed (located in the destroy function, line in file: 273)**
The issue occurs specifically on macOS debug builds when task_handle objects are destroyed, causing memory pool counter mismatches due to debug build object padding and incorrect object size assumptions during polymorphic destruction.

- USD Test Case:

The issue was discovered while running USD's test suite on macOS debug builds, particularly in tests involving parallel processing with task_group and task_handle objects. The assertion failure occurs during test cleanup when task_handle objects are automatically destroyed.
Detailed Race Condition Logs:
When running ctest -R testUsdCatMask -V with OpenUSD, we get the following logs explaining the race condition:
[TBB_CLEANUP] cleanup_list completed, total_removed=40
[TBB_DESTROY] private_list cleaned, removed=40
[TBB_DESTROY] after private cleanup, counter=1
[TBB_DESTROY] public_list exchanged, list=0x1053dfa00
[TBB_CLEANUP] cleanup_list starting, list=0x1053dfa00 thread=0x200d7df00
[TBB_CLEANUP] cleaning object=0x1053dfa00 next=0x1053ff500 count=1
[TBB_CLEANUP] cleaning object=0x1053ff500 next=0x0 count=2
[TBB_CLEANUP] cleanup_list completed, total_removed=2
[TBB_DESTROY] public_list cleaned, removed=2
[TBB_DESTROY] final_private_counter=-1 (will assert if < 0)
[TBB_DESTROY] *** ASSERTION WILL FAIL ***
[TBB_DESTROY] This indicates a race condition in USD's TBB usage
[TBB_DESTROY] private_removed=40 public_removed=2 total_removed=42
[TBB_DESTROY] Expected objects=43 Actual counter=42 Difference=1 (migrated objects)
[TBB_DESTROY] CONCLUSION: 1 object(s) were migrated from other thread pools to this pool
[TBB_DESTROY] These migrated objects caused the counter mismatch
Assertion m_private_counter >= 0 failed (located in the destroy function, line in file: 273)
Detailed description: Private counter may not be less than 0
The logs show that 1 object was "migrated" between thread pools due to incorrect object size calculations during destruction, causing a counter mismatch of 1 object.

# Expected Behavior
Object destruction should use correct object sizes for memory pool operations, maintaining consistent counter values and avoiding assertion failures, regardless of debug build padding.

# Steps To Reproduce
Build oneTBB in debug mode on macOS
Build USD with debug configuration using oneTBB 2022.0.0
Run ctest -R testUsdCatMask -V with OpenUSD
Observe the detailed race condition logs and assertion failure during test cleanup

# Potential Solution:
One possible approach to address this issue would be to:
Make m_allocator protected in task_handle_task to allow derived classes access，change finalize() to pure virtual in the base class to ensure proper polymorphic destruction, implement finalize() override in function_task derived class with correct object size handling.
This would ensure all destruction paths use the correct object sizes through virtual dispatch
eg：
```
diff --git a/include/oneapi/tbb/detail/_task_handle.h b/include/oneapi/tbb/detail/_task_handle.h
index 26212b46..a4d7e9b1 100644
--- a/include/oneapi/tbb/detail/_task_handle.h
+++ b/include/oneapi/tbb/detail/_task_handle.h
@@ -36,11 +36,8 @@ class task_handle_task : public d1::task {
     std::uint64_t m_version_and_traits{};
     d1::wait_tree_vertex_interface* m_wait_tree_vertex;
     d1::task_group_context& m_ctx;
+protected:
     d1::small_object_allocator m_allocator;
 public:
-    void finalize(const d1::execution_data* ed = nullptr) {
-        if (ed) {
-            m_allocator.delete_object(this, *ed);
-        } else {
-            m_allocator.delete_object(this);
-        }
-    }
+    virtual void finalize(const d1::execution_data* ed = nullptr) = 0;
 
     task_handle_task(d1::wait_tree_vertex_interface* vertex, d1::task_group_context& ctx, d1::small_object_allocator& alloc)
         : m_wait_tree_vertex(vertex)
```
And:
```
diff --git a/include/oneapi/tbb/task_group.h b/include/oneapi/tbb/task_group.h
index 393703f4..71cf6ec6 100644
--- a/include/oneapi/tbb/task_group.h
+++ b/include/oneapi/tbb/task_group.h
@@ -84,6 +84,13 @@ class function_task : public task_handle_task  {
     const F m_func;
 
 private:
+    void finalize(const d1::execution_data* ed = nullptr) override {
+        if (ed) {
+            m_allocator.delete_object(this, *ed);
+        } else {
+            m_allocator.delete_object(this);
+        }
+    }
     d1::task* execute(d1::execution_data& ed) override {
         __TBB_ASSERT(ed.context == &this->ctx(), "The task group context should be used for all tasks");
         task* res = task_ptr_or_nullptr(m_func);
```

While the proposed solution addresses the technical issue, making finalize() pure virtual could be considered a breaking change to the header API. According to oneTBB's versioning rules, such changes might require namespace changes to maintain compatibility guarantees. Could you suggest alternative solutions that maintain API compatibility while still addressing the core issue?


Discussion: @fanzhaoxi

Thank you for reporting this issue. 
Unfortunately, we cannot introduce new virtual functions into task_handle_task at this time, as it would break ABI compatibility of the task_handle class.
We've introduced an alternative fix that maintains compatibility in PR #1839.
Could you please verify whether this resolves the issue in your codebase?



Fix (patch):
diff --git a/include/oneapi/tbb/detail/_task_handle.h b/include/oneapi/tbb/detail/_task_handle.h
index 10ea03576f..4f90c59a4f 100644
--- a/include/oneapi/tbb/detail/_task_handle.h
+++ b/include/oneapi/tbb/detail/_task_handle.h
@@ -137,7 +137,15 @@ class task_dynamic_state {
 #endif // __TBB_PREVIEW_TASK_GROUP_EXTENSIONS
 
 class task_handle_task : public d1::task {
-    std::uint64_t m_version_and_traits{};
+    // Pointer to the instantiation of destroy_function_task with the concrete derived type,
+    // used for correct destruction and deallocation of the task
+    using destroy_func_type = void (*)(task_handle_task*, d1::small_object_allocator&, const d1::execution_data*);
+
+    // Reuses the first std::uint64_t field (previously m_version_and_traits) to maintain backward compatibility
+    // The type of the first field remains std::uint64_t to preserve alignment and offset of subsequent member variables.
+    static_assert(sizeof(destroy_func_type) <= sizeof(std::uint64_t), "Cannot fit destroy pointer into std::uint64_t");
+    std::uint64_t m_destroy_func;
+
     d1::wait_tree_vertex_interface* m_wait_tree_vertex;
     d1::task_group_context& m_ctx;
     d1::small_object_allocator m_allocator;
@@ -145,23 +153,29 @@ class task_handle_task : public d1::task {
     std::atomic<task_dynamic_state*> m_dynamic_state;
 #endif
 public:
-    void finalize(const d1::execution_data* ed = nullptr) {
-        if (ed) {
-            m_allocator.delete_object(this, *ed);
+    void destroy(const d1::execution_data* ed = nullptr) {
+        destroy_func_type destroy_func = reinterpret_cast<destroy_func_type>(m_destroy_func);
+        if (destroy_func != nullptr) {
+            // If the destroy function is set for the current instantiation - use it
+            (*destroy_func)(this, m_allocator, ed);
         } else {
-            m_allocator.delete_object(this);
+            // Otherwise, the object was compiled with the old version of the library
+            // Destroy the object and let the memory leak since the derived type is unknown
+            // and the object cannot be deallocated properly
+            this->~task_handle_task();
         }
     }
 
-    task_handle_task(d1::wait_tree_vertex_interface* vertex, d1::task_group_context& ctx, d1::small_object_allocator& alloc)
-        : m_wait_tree_vertex(vertex)
+    task_handle_task(d1::wait_tree_vertex_interface* vertex, d1::task_group_context& ctx,
+                     d1::small_object_allocator& alloc, destroy_func_type destroy_func)
+        : m_destroy_func(reinterpret_cast<std::uint64_t>(destroy_func))
+        , m_wait_tree_vertex(vertex)
         , m_ctx(ctx)
         , m_allocator(alloc)
 #if __TBB_PREVIEW_TASK_GROUP_EXTENSIONS
         , m_dynamic_state(nullptr)
 #endif
     {
-        suppress_unused_warning(m_version_and_traits);
         m_wait_tree_vertex->reserve();
     }
 
@@ -227,10 +241,10 @@ class task_handle_task : public d1::task {
 };
 
 class task_handle {
-    struct task_handle_task_finalizer_t{
-        void operator()(task_handle_task* p){ p->finalize(); }
+    struct task_handle_task_deleter {
+        void operator()(task_handle_task* p){ p->destroy(); }
     };
-    using handle_impl_t = std::unique_ptr<task_handle_task, task_handle_task_finalizer_t>;
+    using handle_impl_t = std::unique_ptr<task_handle_task, task_handle_task_deleter>;
 
     handle_impl_t m_handle = {nullptr};
 public:
diff --git a/include/oneapi/tbb/task_group.h b/include/oneapi/tbb/task_group.h
index a0ca252e94..df94e082da 100644
--- a/include/oneapi/tbb/task_group.h
+++ b/include/oneapi/tbb/task_group.h
@@ -85,6 +85,16 @@ class function_task : public task_handle_task  {
     const F m_func;
 
 private:
+    static void destroy_function_task(task_handle_task* p, d1::small_object_allocator& alloc,
+                                      const d1::execution_data* ed)
+    {
+        if (ed) {
+            alloc.delete_object(static_cast<function_task*>(p), *ed);
+        } else {
+            alloc.delete_object(static_cast<function_task*>(p));
+        }
+    }
+
     d1::task* execute(d1::execution_data& ed) override {
         __TBB_ASSERT(ed.context == &this->ctx(), "The task group context should be used for all tasks");
         task* next_task = task_ptr_or_nullptr(m_func);
@@ -99,7 +109,7 @@ class function_task : public task_handle_task  {
             next_task = successor_task;
         }
 #endif
-        finalize(&ed);
+        this->destroy(&ed);
         return next_task;
     }
     d1::task* cancel(d1::execution_data& ed) override {
@@ -109,14 +119,14 @@ class function_task : public task_handle_task  {
         // Should cancel() be called directly instead?
         task_ptr = this->complete_and_try_get_successor();
 #endif
-        finalize(&ed);
+        this->destroy(&ed);
         return task_ptr;
     }
 public:
     template<typename FF>
     function_task(FF&& f, d1::wait_tree_vertex_interface* vertex, d1::task_group_context& ctx, d1::small_object_allocator& alloc)
-        : task_handle_task{vertex, ctx, alloc},
-          m_func(std::forward<FF>(f)) {}
+        : task_handle_task{vertex, ctx, alloc, destroy_function_task}
+        , m_func(std::forward<FF>(f)) {}
 };
 
 #if __TBB_PREVIEW_TASK_GROUP_EXTENSIONS