I believe setting task to None quietly will not work, precisely because listening action depends on the trait change notification to disconnect listeners to task's extended properties, if used in the enabled_when/visible_when or other extended trait Properties. However, it may not be a problem in this case because actions (menu items and toolbar buttons) are destroyed mostly after the task window is closed, in which case it it unlikely that any of them will be triggered.
However, for confirmation, let me try and verify it tomorrow.