Avoid double destroy when handle has multiple destroy methods
(ex: destroy() and cancel()).  Fixes #17763.

Thanks Benjamin Santalucia for patch!