Be safe about removing minion ids

Closes #16782
Merge pull request #16791 from cachedout/issue_16782

Be safe about removing minion ids