Bug: Some broadcasts cannot be cleared from memory storage
Status: Resolved | Priority: Major | Resolution: Not A Problem
Created: 02/Aug/24 02:41
Affects Version/s: 3.4.0

Please apply this patch([^SPARK-49091.patch]) to reproduce this issue. This issue may cause driver memory leak.
 !driver heap.png|thumbnail!

This issue was introduced by SPARK-41914.
Before SPARK-41914:
{noformat}
[info] BroadcastCleanerSuite:
10:30:16.228 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 0, names: 
[info] - Test broadcast cleaner (1 minute, 4 seconds)
10:31:21.552 WARN org.apache.spark.sql.BroadcastCleanerSuite:
{noformat}

After SPARK-41914:
{noformat}
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 8, names: broadcast_0_piece0, broadcast_0, broadcast_1_piece0, broadcast_2_piece0, broadcast_2, broadcast_1, broadcast_3_piece0, broadcast_3
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
entries size: 2, names: broadcast_1_piece0, broadcast_1
[info] *** Test still running after 4 minutes, 58 seconds: suite name: BroadcastCleanerSuite, test name: Test broadcast cleaner. 
...
{noformat}