Only fix IE removed node bugs if a remove callback is specified.