Make sure we only decrement n_object when we truly deallocate
an object.