Un-own objects when a handle is assigned.