Destroy ref member when destroy user