Repository: dotnet/runtime
Pull request #114545: Added missing clearing of pooled arrays
Author: login: hendriklhf
Labels: name: area-System.Collections
name: community-contribution
description: Indicates that the PR has been added by a community member
Created: 2025-04-11T15:51:03Z | Merged: 2025-04-24T00:20:05Z

I found a few places where arrays were returned to the pool but not cleared, which would undesirably keep remaining objects in the array alive.
The check for `typeof(T).IsPrimitive` is of course not 100% correct, but, if I'm not mistaken, `T` is never a non-primitive unmanaged type in all current usages.


Diff:
diff --git a/src/libraries/System.Diagnostics.DiagnosticSource/src/System.Diagnostics.DiagnosticSource.csproj b/src/libraries/System.Diagnostics.DiagnosticSource/src/System.Diagnostics.DiagnosticSource.csproj
index bbd90f7f5ee0ec..9fb48dbfd83a6f 100644
--- a/src/libraries/System.Diagnostics.DiagnosticSource/src/System.Diagnostics.DiagnosticSource.csproj
+++ b/src/libraries/System.Diagnostics.DiagnosticSource/src/System.Diagnostics.DiagnosticSource.csproj
@@ -151,6 +151,7 @@ System.Diagnostics.DiagnosticSource</PackageDescription>
     <Reference Include="System.Collections" />
     <Reference Include="System.Collections.Concurrent" />
     <Reference Include="System.Diagnostics.Tracing" />
+    <Reference Include="System.Linq" />
     <Reference Include="System.Memory" />
     <Reference Include="System.Runtime" />
     <Reference Include="System.Runtime.InteropServices" />
diff --git a/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs b/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
index f446960d647739..42fbaeca7cbc71 100644
--- a/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
+++ b/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
@@ -3,7 +3,7 @@
 
 using System.Buffers;
 using System.Collections.Generic;
-using System.Runtime.CompilerServices;
+using System.Linq;
 
 namespace System.Diagnostics.Metrics
 {
@@ -32,7 +32,7 @@ public Measurement(T value)
         /// <param name="tags">The <see cref="KeyValuePair{TKey, TValue}"/> tags associated with the measurement.</param>
         public Measurement(T value, IEnumerable<KeyValuePair<string, object?>>? tags)
         {
-            _tags = ToArray(tags);
+            _tags = tags?.ToArray() ?? Instrument.EmptyTags;
             Value = value;
         }
 
@@ -98,67 +98,5 @@ public Measurement(T value, in TagList tags)
         /// Gets the value of the measurement.
         /// </summary>
         public T Value { get; }
-
-        // Private helper to copy IEnumerable to array. We have it to avoid adding dependencies on System.Linq
-        private static KeyValuePair<string, object?>[] ToArray(IEnumerable<KeyValuePair<string, object?>>? tags)
-        {
-            if (tags is null)
-                return Instrument.EmptyTags;
-
-            KeyValuePair<string, object?>[] result;
-
-            // When the input is a collection, we can allocate a correctly sized array and copy directly into it.
-            if (tags is ICollection<KeyValuePair<string, object?>> collection)
-            {
-                int items = collection.Count;
-
-                if (items == 0)
-                    return Instrument.EmptyTags;
-
-                result = new KeyValuePair<string, object?>[items];
-                collection.CopyTo(result, 0);
-                return result;
-            }
-
-            // In any other case, we must enumerate the input.
-            // We use a pooled array as a buffer to avoid allocating until we know the final size we need.
-            // This assumes that there are 32 or fewer tags, which is a reasonable assumption for most cases.
-            // In the worst case, we will grow the buffer by renting a larger array.
-            KeyValuePair<string, object?>[] array = ArrayPool<KeyValuePair<string, object?>>.Shared.Rent(32);
-            int count = 0;
-            int length = array.Length;
-
-            foreach (KeyValuePair<string, object?> item in tags)
-            {
-                if (count == length)
-                    Grow(ref array, ref length);
-
-                array[count++] = item;
-            }
-
-            if (count == 0)
-            {
-                ArrayPool<KeyValuePair<string, object?>>.Shared.Return(array);
-                return Instrument.EmptyTags;
-            }
-
-            result = new KeyValuePair<string, object?>[count];
-            array.AsSpan().Slice(0, count).CopyTo(result.AsSpan());
-
-            // Note that we don't include the return of the array inside a finally block per the established guidelines for ArrayPool.
-            // We don't expect the above to throw an exception, but if it does it would be infrequent and the GC will collect the array.
-            ArrayPool<KeyValuePair<string, object?>>.Shared.Return(array);
-            return result;
-
-            [MethodImpl(MethodImplOptions.AggressiveInlining)]
-            static void Grow(ref KeyValuePair<string, object?>[] array, ref int length)
-            {
-                KeyValuePair<string, object?>[] newArray = ArrayPool<KeyValuePair<string, object?>>.Shared.Rent(length * 2);
-                array.CopyTo(newArray, 0);
-                ArrayPool<KeyValuePair<string, object?>>.Shared.Return(array);
-                array = newArray;
-                length = array.Length;
-            }
-        }
     }
 }
diff --git a/src/libraries/System.Diagnostics.DiagnosticSource/tests/MetricsTests.cs b/src/libraries/System.Diagnostics.DiagnosticSource/tests/MetricsTests.cs
index 05bd3aed1bd0e1..3a83e2ba346257 100644
--- a/src/libraries/System.Diagnostics.DiagnosticSource/tests/MetricsTests.cs
+++ b/src/libraries/System.Diagnostics.DiagnosticSource/tests/MetricsTests.cs
@@ -26,6 +26,10 @@ public void MeasurementConstructionTest()
                 Assert.Equal(i, measurement.Value);
                 TagListTests.ValidateTags(new TagList(measurement.Tags), tagsArray);
 
+                measurement = new Measurement<int>(i, tagsArray.AsEnumerable());
+                Assert.Equal(i, measurement.Value);
+                TagListTests.ValidateTags(new TagList(measurement.Tags), tagsArray);
+
                 measurement = new Measurement<int>(i, tagsArray);
                 Assert.Equal(i, measurement.Value);
                 TagListTests.ValidateTags(new TagList(measurement.Tags), tagsArray);
diff --git a/src/libraries/System.Private.CoreLib/src/System/Buffers/ArrayPool.cs b/src/libraries/System.Private.CoreLib/src/System/Buffers/ArrayPool.cs
index 7ea932f3bd8037..9f6fcc8ac076bc 100644
--- a/src/libraries/System.Private.CoreLib/src/System/Buffers/ArrayPool.cs
+++ b/src/libraries/System.Private.CoreLib/src/System/Buffers/ArrayPool.cs
@@ -97,5 +97,11 @@ public static ArrayPool<T> Create(int maxArrayLength, int maxArraysPerBucket) =>
         /// if it's determined that the pool already has enough buffers stored.
         /// </remarks>
         public abstract void Return(T[] array, bool clearArray = false);
+
+        internal void Return(T[] array, int lengthToClear)
+        {
+            array.AsSpan(0, lengthToClear).Clear();
+            Return(array);
+        }
     }
 }
diff --git a/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.cs b/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.cs
index 8c84415aaea093..dbdf8b157f2b68 100644
--- a/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.cs
+++ b/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.cs
@@ -165,7 +165,24 @@ public void Dispose()
             if (toReturn != null)
             {
                 _arrayFromPool = null;
+
+#if SYSTEM_PRIVATE_CORELIB
+                if (RuntimeHelpers.IsReferenceOrContainsReferences<T>())
+                {
+                    ArrayPool<T>.Shared.Return(toReturn, _pos);
+                }
+                else
+                {
+                    ArrayPool<T>.Shared.Return(toReturn);
+                }
+#else
+                if (!typeof(T).IsPrimitive)
+                {
+                    Array.Clear(toReturn, 0, _pos);
+                }
+
                 ArrayPool<T>.Shared.Return(toReturn);
+#endif
             }
         }
 
@@ -200,7 +217,23 @@ private void Grow(int additionalCapacityRequired = 1)
             _span = _arrayFromPool = array;
             if (toReturn != null)
             {
+#if SYSTEM_PRIVATE_CORELIB
+                if (RuntimeHelpers.IsReferenceOrContainsReferences<T>())
+                {
+                    ArrayPool<T>.Shared.Return(toReturn, _pos);
+                }
+                else
+                {
+                    ArrayPool<T>.Shared.Return(toReturn);
+                }
+#else
+                if (!typeof(T).IsPrimitive)
+                {
+                    Array.Clear(toReturn, 0, _pos);
+                }
+
                 ArrayPool<T>.Shared.Return(toReturn);
+#endif
             }
         }
     }


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: 

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.




state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-11T15:51:24Z
user: login: stephentoub
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T16:03:49Z
user: login: tarekgh
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T19:09:50Z
user: login: stephentoub
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T19:19:07Z
user: login: tarekgh
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T21:52:48Z
user: login: tarekgh
body: LGTM. Thanks!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-22T18:14:54Z

Review comments:
diff_hunk: @@ -165,6 +165,16 @@ public void Dispose()
             if (toReturn != null)
             {
                 _arrayFromPool = null;
+
+#if NET || NETSTANDARD2_1_OR_GREATER
+                if (RuntimeHelpers.IsReferenceOrContainsReferences<T>())
+#else
+                if (!typeof(T).IsPrimitive)
path: src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.cs
user: login: Copilot
body: The fallback check using !typeof(T).IsPrimitive may not accurately determine if T contains references for value types. Please add a clarifying comment that explains this assumption, given that in current usages T is always a primitive unmanaged type.
created_at: 2025-04-11T15:51:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -200,6 +210,15 @@ private void Grow(int additionalCapacityRequired = 1)
             _span = _arrayFromPool = array;
             if (toReturn != null)
             {
+#if NET || NETSTANDARD2_1_OR_GREATER
+                if (RuntimeHelpers.IsReferenceOrContainsReferences<T>())
+#else
+                if (!typeof(T).IsPrimitive)
path: src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.cs
user: login: Copilot
body: Similar to the Dispose method, the Grow method uses the !typeof(T).IsPrimitive check as a fallback. Consider adding a comment here to clarify that this condition is acceptable because T is ensured to be a primitive unmanaged type in all current contexts.
created_at: 2025-04-11T15:51:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -145,6 +145,8 @@ public Measurement(T value, in TagList tags)
             result = new KeyValuePair<string, object?>[count];
             array.AsSpan().Slice(0, count).CopyTo(result.AsSpan());
 
+            Array.Clear(array, 0, count);
path: src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
user: login: stephentoub
body: @tarekgh, the comment on this method says:
"Private helper to copy IEnumerable to array. We have it to avoid adding dependencies on System.Linq"
Why don't we want to just use LINQ's ToArray here? We're concerned about loading System.Linq.dll?
created_at: 2025-04-11T16:03:48Z
author_association: MEMBER
diff_hunk: @@ -145,6 +145,8 @@ public Measurement(T value, in TagList tags)
             result = new KeyValuePair<string, object?>[count];
             array.AsSpan().Slice(0, count).CopyTo(result.AsSpan());
 
+            Array.Clear(array, 0, count);
path: src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
user: login: tarekgh
body: Yes, IIRC long ago some partners were developing micro services and worked hard to reduce any new dependencies. We initially added this dependency on Linq and then later removed it because of that.

Cc @noahfalk 
created_at: 2025-04-11T19:09:50Z
author_association: MEMBER
diff_hunk: @@ -145,6 +145,8 @@ public Measurement(T value, in TagList tags)
             result = new KeyValuePair<string, object?>[count];
             array.AsSpan().Slice(0, count).CopyTo(result.AsSpan());
 
+            Array.Clear(array, 0, count);
path: src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
user: login: stephentoub
body: Do we know if that's still relevant? If you're in this part of the code, you're dealing with an IEnumerable that's _not_ a collection. I'd be a bit surprised if such consuming code was part of an application that wasn't already using System.Linq.dll somewhere. (It's also not a separate dependency requiring distribution with the app, as it's part of any .NET that would be targeted.)
created_at: 2025-04-11T19:19:07Z
author_association: MEMBER
diff_hunk: @@ -145,6 +145,8 @@ public Measurement(T value, in TagList tags)
             result = new KeyValuePair<string, object?>[count];
             array.AsSpan().Slice(0, count).CopyTo(result.AsSpan());
 
+            Array.Clear(array, 0, count);
path: src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/Metrics/Measurement.cs
user: login: tarekgh
body: We can try to change it and look if we see any complaints again. 

@noahfalk let us know if you have more context or concern depending on Linq.
created_at: 2025-04-11T21:52:48Z
author_association: MEMBER

PR comments:
user: login: hendriklhf
created_at: 2025-04-11T15:52:59Z
body: @dotnet-policy-service agree
author_association: CONTRIBUTOR
user: login: tannergooding
created_at: 2025-04-11T16:08:59Z
body: Should we rather be ensuring that `ArrayPool` itself clears the returned array for any `RuntimeHelpers.IsReferenceOrContainsReferences<T>()`, much like we do for `List<T>` and other scenarios?

This seems more like a general issue with potentially keeping objects alive and not something that should be handled per callsite prior to the return.
author_association: MEMBER
user: login: stephentoub
created_at: 2025-04-11T16:13:37Z
body: > Should we rather be ensuring that ArrayPool itself clears the returned array

It doesn't know how much to clear; it would have to clear the whole thing, which is what it does in Return when the clearArray parameter is true.
author_association: MEMBER
user: login: tannergooding
created_at: 2025-04-11T16:28:45Z
body: > It doesn't know how much to clear; it would have to clear the whole thing

Right, but it seems like that's better/cheaper than potentially holding onto other references for an unknown amount of time.

It really seems like there should be an analyzer and/or some overload of `Return` that makes this significantly more convenient and less problematic.

For example, some `void Return(T[] array, int lengthToClear)` + an analyzer for a non unmanaged type `T` could help catch issues like this PR is fixing while reducing code duplication.
author_association: MEMBER
user: login: tarekgh
created_at: 2025-04-21T18:21:51Z
body: It seems to me @tannergooding suggestion is a better solution here to avoid the callers of the `ArrayPool` clearing the list before returning it. @hendriklhf can you try it? and we can open tracking issue for having an analyzer for that.
author_association: MEMBER
user: login: tarekgh
created_at: 2025-04-21T18:24:35Z
body: Additionally, @hendriklhf if you interested to apply the change having System.Diagnostics.DiagnosticSource take dependency on Linq and then remove the code of `Measurement<T>.ToArray()` replacing the calling sites with the `Linq.ToArray`? Please let me know if you can't do so I can help with it.

CC @noahfalk 
author_association: MEMBER
user: login: hendriklhf
created_at: 2025-04-22T17:29:18Z
body: > Additionally, @hendriklhf if you interested to apply the change having System.Diagnostics.DiagnosticSource take dependency on Linq and then remove the code of `Measurement<T>.ToArray()` replacing the calling sites with the `Linq.ToArray`?

Done 👍 

> It seems to me @tannergooding suggestion is a better solution here to avoid the callers of the `ArrayPool` clearing the list before returning it. @hendriklhf can you try it? and we can open tracking issue for having an analyzer for that.

Yes, added the `Return(T[], int)` overload on `ArrayPool` as an internal for that.
author_association: CONTRIBUTOR
user: login: hendriklhf
created_at: 2025-04-22T19:57:15Z
body: Now that I think about it, it might make more sense to swap which `Return` calls which, so there is no new place introduced where the array is cleared and is also only cleared if it can be returned. Would just require overriding the new `Return` in the two inheriting types and moving the return logic to it. But I can also leave it as it is, whatever you prefer.
But if it will become public with an analyzer, couldn't be abstract, as it would be breaking, right? So might have to stay as it is right now. 🤔 

```c#
public override void Return(T[] array, bool clearArray = false)
{
    ArgumentNullException.ThrowIfNull(array);
    Return(array, clearArray ? array.Length : 0);
}

internal override void Return(T[] array, int lengthToClear)
{
    ...
}
```
author_association: CONTRIBUTOR
user: login: tarekgh
created_at: 2025-04-22T20:14:02Z
body: @hendriklhf Instead of introducing a new internal method `internal override void Return(T[] array, int lengthToClear)` within the `ArrayPool` class, I'm considering implementing it as an extension helper method instead. This approach would enable its use in libraries targeting down-level platforms (such as .NET Standard 2.0 and .NET Framework). Eventually we may consider a public method in ArrayPool. 
author_association: MEMBER
user: login: tarekgh
created_at: 2025-04-24T00:19:48Z
body: I am merging this PR. We can consider moving ` Return(T[] array, int lengthToClear)` to an extension method if we see a need to use it in the down-level targeting code. 
author_association: MEMBER