Properly despawn pets on world change.