Bug: Decommission logs too frequent when waiting migration to finish
Status: In Progress | Priority: Major | Resolution: 
Created: 18/Aug/21 12:55
Affects Version/s: 3.1.0

{code:java}
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
...{code}
It takes some time to migrate data (shuffle or rdd). Logging per second is too frequent.