Fix bugs in setup and when reference object is deleted