Clear dirty trackers if component was removed