Update validity when removing form elements.

Fixes a few bugs with validation updates when detaching validators/controls.