Handle package remove in a separate thread