switch: Fix attaching/detaching ports.