[SPARK-3709] Executors don't always report broadcast block removal properly back to the driver (for branch-1.1)

Author: Reynold Xin <rxin@apache.org>

Closes #2591 from rxin/SPARK-3709-1.1 and squashes the following commits:

ab99cc0 [Reynold Xin] [SPARK-3709] Executors don't always report broadcast block removal properly back to the driver

(cherry picked from commit a8c6e82de31cb09b7e7e37caf4e52c9e10185b63)
Signed-off-by: Reynold Xin <rxin@apache.org>