Use a list of "dirty" components for deleted components in ComponentMgr.
Clean them up during update.