Refactor external dependencies to clean up after themselves

[Fixes #44207999]