Don't ptr_dtor when observed value is a reference