Make sure the reference to the connector is removed (see #10333)

This PR fixes the second part of https://trac.openmicroscopy.org.uk/ome/ticket/10333