Fix a bug that TaskMonitor will remove task unfinised.