Setup the talloc destructors correctly